Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

ESP (Enterprise Services Platform) is an HPCC Systems component that hosts web services functions. It’s a container for running services. ESP is what provides you with the ability to write and run web services in HPCC Systems, support revenue based products and use the dynamic ESDL engine which gives you flexibility and ease of use. A couple of examples of ESP services are WsECL, which allows you to interface to published queries on Roxie or Thor, and ECL Watch which is our web-based query execution, monitoring and file management interface. ESP supports a number of protocols including, SOAP, XML, HTTP, REST and JSON.  

Such a powerful component of the HPCC Systems stack requires refined instrumentation techniques and the current goal is to add execution profiling for ESP services at the method level. Key to taking advantage of this feature is to define the execution timing buckets used to measure the execution time distribution for each method. Unfortunately, this is something that needs to be set manually as the histogram metric being used does not adjust the buckets based on measurement data (nor should it). This means that the bucket limits need to be tailored at a minimum for each service and probably individually for each methodIn this context, having a command line tool that can be used to call any service within ESP would be extremely useful. Such command line tool could be used, for instance, to test the service APIs, to learn what the APIs are, and for users to automate tasks that do not currently have specialized implementations within the other HPCC Systems command line tools.

The objective of this project is to provide starting point values for the execution timing buckets used to measure the execution time distribution of each method. develop a command line tool with such capabilities. The tool would take the XML files generated from the SCM files for the ESP services and use them to dynamically configure a tool for calling those services. The syntax supported by the command line tool would be something like "hpcc <service> <method> <target> [options]",  where the options parameter are all of the form "--<method-parameter>=<value>". For example:

Code Block
languagepy
hpcc WsDfu DFUSearch myEspUrl --showExample=yes

To accomplish this, the following process needs to be followed:

...

Mentor

Ken Rowland
Kenneth.Rowland@lexisnexisrisk.com

Backup Mentor: 

TBD

Skills needed
  • General knowledge of Linux and web services development
  • Self-motivated to learn about HPCC Systems and monitoring tools such as Prometheus (guidance will be provided)
  • Willing to explore new technologies
  • Basic programming skills such as Unix shell (bash), WSDL, Python, etc
Other resources