How To... Videos - ECL Tips and Tricks

These videos were created by Allan Wrobel (Consulting Software Engineer). Allan has been working in the IT industry his entire adult life and has been an employee with LexisNexis Risk Solutions Group since 2010.

How to...
Use Macros
Use ECL Macros Session 1- A worked through case A explanation of ECL (Enterprise Control Language) MACROS using a real life case study. Compare attributes between two key build where the number of attributes is so large that using MACROS is the only solution.
Use ECL Macros Session 2- A worked through case Constructing a new output layer.
Use ECL Macros Session 3- A worked through case Generating an email using ECL to act on the information discovered in the previous session
Be clear about the context in which the ECL FUNCTIONMACROs run Demonstrates that ECL FUNCTIONMACROs compile in the context of the caller and not in the context of the ECL in which the FUNCTIONMACRO resides. There is, at yet, no equivalent of C++ of this in ECL that enables access to an instances variables/state.
ECL #EXPORT and #EXPORTXML Processing record structures at compile time using MACROS
Understand the idiosyncrasies of ECL MACROs and #EXPAND Where #EXPAND can and can't be used in a macro.
Use ECL built-in Functions
PROJECT Converting records in a dataset from one format to another.
HOW ECL PROJECT is different from other built-ins The ECL PROJECT built-in, unlike other built-in, accepts single Record as it's input. Other built-ins require datasets.
TABLE (Vertical Slice) Taking an import dataset in one format and generating an output dataset in another format.
TABLE (CrossTab) Getting statistics on input datasets grouped by fields within the dataset.
Use Aggregates on sub sets of data in a cross tab report Find out about the difference between the COUNT operator and other operators used in the context of ECL cross-tab TABLE function.
ITERATE Processes all records in the recordset one pair of records at a time, performing the transform function on each pair in turn.
PROCESS (Part 1) Processes all records in the recordset one pair of records at a time, performing the datasettransform function on each pair of records in turn.
PROCESS (Part 2) Processes all records in the recordset one pair of records at a time, performing the datasettransform function on each pair of records in turn.
GRAPH A useful and underused function allowing you to search a dataset with different attributes/dimensions which have multiple values. Code examples for this tip can be found in the related HPCC Systems Tips and Trick Forum post.
NORMALIZE (COUNT Variant) This is the simplest of the 2 NORMALIZE functions. This variant is used to call a TRANSFORM repeatedly for each record in a dataset. An interesting use of the NORMALIZE function can also be found in the HPCC Systems ECL Programmer's Guide.
NORMALIZE (Child Dataset Extraction) This variant of the NORMALIZE function is used to extract child datasets from an hierarchical dataset.
NORMALIZE (Additional Information) Find out more about some additional items such as using child datasets inside child datasets and the behaviour of SKIP.
DENORMALIZE (Single Record Presentation Variant) The inverse to NORMALIZE. This variant covers where inputs are presented singularly to the TRANSFORM. This video also mentions some subtleties of the behaviour of this function.
DENORMALIZE (GROUP Presentation Variant) The inverse to NORMALIZE for child datasets. This variant covers when Child records are presented as DATASETs to the TRANSFORM.
ECL DICTIONARY (Examples of use) DICTIONARY is a fast lookup for very large tables/datasets. This video expands on the examples included in the HPCC Systems ECL Language Reference Guide.
Working with distributed data (TABLE, ROLLUP and AGGREGATE) How to mine data using the HPCC Systems Architecture.
ROLLUP (Part 1) Allows you to deduplicate records with the additional facility to capture information from input records while deduplicating the same.
ROLLUP (Part 2) Follows on from part 1 in this series covering how to preserve the information from one record to the next between iterations and while deduping.
DEDUP The DEDUP function evaluates the recordset for duplicate records, as defined by the condition parameter, and returns a unique return set
Be wary of using the Compiler hint DISTRIBUTED in ECL DISTRIBUTED is not to be confused with the DISTRIBUTE command, that actually distributes data-sets.
Find a Better way to optimise out file re-distributions using DISTRIBUTED Learn the best way to use the ECL compiler hint DISTRIBUTED and how to use DISTRIBUTED safely to optimise out unnecessary re-distributions of HPCC Systems logical files.
Learn more tips and tricks
Understand the differences Between Build and definition files in HPCC Systems To anyone new to HPCC Systems and the ECL language, the way HPCC Systems behaves differently with the two concepts of Actions and Definitions can be very confusing. Learn about the behavior used with both and find out more about why HPCC Systems runs this way.
Explicit return types from functions Find out how defaulting return types form functions degrades the compilers ability to correctly report errors.
Understand the Distribution of data on HPCC Systems, with an example Describes the Distribution of Logical files onto multiple nodes of an HPCC Systems environment. Also find out more about the measure of SKEW from an even distribution as defined for HPCC Systems.
Use ECL/HPCC Systems Events, Including Scheduling Events Demonstration of creating, fielding and firing events in HPCC Systems. Also includes some additional notes on scheduling events (CRON), including reading the output of one workunit in another.

How To... Videos - ECL Tips and Tricks

How to...

Use Macros

Use ECL built-in Functions

Learn more tips and tricks