Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
  • ECL concepts
  • Practical ECL with flight data

    Date and time of the event

    23-26th January 2023

    Location

    New College Of Florida, FL, 5800 Bay Shore Rd, Sarasota,  34243

    Cost

    Free

    Hosted by

    New College of Florida


    Open to NCF students

    Registration 

    n/a

    Overview

    HPCC Systems is a distributed big data technology used by LexisNexis Risk Solutions Group.  It was, in fact, created by LNRS long before Hadoop came along and it still serves as the core technology behind a $3B/year business.

    In this workshop you will be introduced to the HPCC Systems technology and to the data flow language used to program on it, ECL.  The data you will be examining is a detailed global schedule of airline flights.  Each record contains not only takeoff/landing places and times, but details regarding the plane, meals, and crew are included as well.

    Learn

    Monday, January 23

    • Introduction to HPCC Systems
    • Introduction to ECL
    • Student IDE setup and configuration
    • Data Discovery — getting acquainted with the flight data
    • Review data dictionary
    • Profiling the actual data
    • Manual inspection via simple ECL

    Tuesday, January 24

    Practical ECL with flight data (cont'd)
    • Intro and welcome – Bob, Richard, Hugo

    • Internship info session – Hugo 

    • Lecture on Data Evaluation, Crosstab, Profiling – Richard/Bob

    • Lab Period – Bob

    • 1:1 Internship/Career discussion - Hugo 

    Tuesday, January 24

    • Lecture on PROJECT, Sequencing, Standardization – Richard/Bob

    • Lab Period – Bob

    • 1:1 Internship/Career discussion - Hugo   

    Wednesday, January 25

  • ECL concepts (cont'd)
  • Data SourceTBA <add the link>Workshop Trainers

    Bob Foreman 

    • Lecture on Data Delivery (ROXIE) – Richard/Bob

    • Lab Period – Bob

    • 1:1 Internship/Career discussion - Hugo   

    Thursday, January 26

    • ECL concepts (cont'd)
    • Practical ECL with flight data (cont'd)
    • Lecture on ECL Cookbook, XML/JSON Techniques - Richard/Bob

    • Lab Period - Bob

    • 1:1 Internship/Career discussion - Hugo   

    Data Source

    N/A

    Workshop Trainers

    Image Added


    Richard Taylor - Bob Foreman - Hugo Watanuki

    What you need

    A laptop

    Pre-work

    The following are recommended but optional:

    Learn about ECL - HPCC Systems ECL Concepts videos

    Learn about ECL - Documentation: https://hpccsystems.com/training/documentation

    • Learning ECL

    • ECL IDE and Client Tools

    • Tutorials

    Install IDE

    VS Code - Supported operating systems

    • macOS

    • Windows

    • CentOS 6 and 7

    • Ubuntu 16.04, 18.04, and 19.04

    Installation instructions: https://

    wiki

    hpccsystems.

    hpccsystems

    atlassian.

    com

    net/wiki/display/hpcc/VS+Code+and+HPCC+Systems+Installation+Cheat+Sheet

    ECL IDE -Supported operating systems: Windows

    Internship Opportunities

    Find out more about the HPCC Systems intern program

    What can I do to prepare? <review for accuracy>

    We have many resources available for you to use to familiarise yourself with our technology and how to use it. If you are new to us, then take some time to find out about HPCC Systems and what we do. Find out how HPCC Systems workswhat ECL is and take a look at what goes on in our community.

    • Watch a quick overview video about HPCC Systems

    • Download the HPCC Systems VM. Select the operating systems you are using first and then check the VM download. Follow the installation guide instructions.

      • Note:  We will be using a cloud-based HPCC Systems cluster for the workshop.

    • You can use your preferred editor to write code but we do have our own, a Windows-based ECL IDE which you can download. On the download page, select Gold and under Operating System, select Windows. Download both the ECL IDE and Client Tools.

    • VS Code is a good code editor if you don't use Windows.  Installation is slightly more complicated see below and also this cheat sheet:

      1. Download and install VS Code from here if you don't already have it installed.

      2. Download the HPCC Systems Client Tools from here.

        1. Choose your operating system from the popup list.

        2. Choose the appropriate "Client Tools" option for your operating system.  Make sure only one checkbox on the entire page is selected.

        3. Download and install.

      3. Launch VS Code, then search for and install the extension named "ECL (Enterprise Control Language) support for Visual Studio Code".

    • Once you’re up and running, try out a few examples from the installation guide and tutorials.

    • Learn some ECL. This is the language used to write queries. It's easy to use, try it for yourself. Read the documentation or take a training course.

    • Take a look at some video tutorials

    • Take a look at the information and training examples in this GitHub repository. In particular, please look at the Taxi_Tutorial where you will find the DataSeers contribution which provides examples showing some basic ECL functionality in action.

    • Take a look at our Machine Learning Documentation and Sources.