Create a new hpcc command line tool

This project is available as a student work experience opportunity with HPCC Systems. Curious about other projects we are offering? Take a look at our Ideas List.

Student work experience opportunities also exist for students who want to suggest their own project idea. Project suggestions must be relevant to HPCC Systems and of benefit to our open source community. 

Find out about the HPCC Systems Summer Internship Program.

Project Description

ESP (Enterprise Services Platform) is an HPCC Systems component that hosts web services functions. It’s a container for running services. ESP is what provides users and organizations with the ability to write and run web services in HPCC Systems, support revenue based products, and can leverage the dynamic ESDL engine for flexibility and ease of use. A couple of examples of ESP services are WsECL, which allows you to interface to published queries on Roxie or Thor, and ECL Watch which is the web-based query execution, monitoring and file management interface. ESP supports a number of protocols including, SOAP, XML, HTTP, REST and JSON.  

In this context, having a command line tool that can be used to call any service within ESP would be extremely useful. Such command line tool could be used, for instance, to test the service APIs, to learn what the APIs are, and for users to automate tasks that do not currently have specialized implementations within the other HPCC Systems command line tools.

The objective of this project is to develop a command line tool with such capabilities. The tool would take the XML files generated from the ECM files for the ESP services and use them to dynamically configure a tool for calling those services. The syntax supported by the command line tool would be something like:

hpcc <service> <method> <target> [options]

where the options parameter are all of the form "--<method-parameter>=<value>". For example:

hpcc WsDfu DFUSearch myEspUrl --showExample=yes

To accomplish this, the following requirements must be considered:

• It should be completely generalized with no special casing - configured purely by the XML files.
• It should be integrated with the Linux command completion and provide context sensitive completion.
• It should have built in help, also generated from the XML files.
• Extension: The output could be controlled to use JSON/XML and it could use XPATH to extract values from the result.
• It would initially only support scalar arguments. 

Completion of this project involves:

• Learning general HPCC Systems architecture.
• Learning ESP services framework.
• Development of a functional command line tool as specified above.
• Creation of test cases for ESP services and specific methods.
• Creation of documentation including a GitHub repository for the project.
• A blog, a recorded presentation, and a poster artifact about your project (see examples from previous years here).