HPCC Systems Cloud Deployment with Various Helm Chart Configurations

Owned by Hugo Watanuki
Last updated: Jul 17, 2023
2 min read

This project is already taken and is no longer available for the 2023 HPCC Systems Intern Program

This project is available as a student work experience opportunity with HPCC Systems. Curious about other projects we are offering? Take a look at our Ideas List.

Student work experience opportunities also exist for students who want to suggest their own project idea. Project suggestions must be relevant to HPCC Systems and of benefit to our open source community. 

Find out about the HPCC Systems Summer Internship Program.

Project Description

HPCC Systems used to have a pre-packaged Virtual Machine for usage with VirtualBox or Hyper-V which was very helpful for new users to experiment and learn HPCC Systems as well for troubleshooting purposes. Since HPCC Systems has now moved to the cloud we now support several alternative local cloud environments such as Docker Desktop, Minikube, etc. However, all these alternatives require users to setup a local Kubernetes cluster as a prerequisite before deploying and configuring the cloud native version of HPCC Systems.

K3D is a lightweight wrapper to run K3S (Rancher Lab's minimal Kubernetes distribution) in docker which makes it very easy to create single and multi-node K3S clusters in docker. In this project the student will be responsible for configuring K3D on Unix (Linux or Mac OS) and Azure with following HPCC Cloud scenarios according to Helm settings:

  • Dali only cluster

  • Dropzone cluster

  • hthor only cluster

  • Roxie only cluster

  • Thor only cluster

  • Logging

  • Monitoring

  • TLS support

  • Cluster for ECL VSCode Notebook support

Completion of this project involves:

  •  Learning Docker, Kubernetes, Helm, Docker Desktop, K3D/K3D and Azure

  • Learning HPCC System Cloud development, specially various HELM configuration and ECL VSCode Notebook

  • HPCC Cloud scenarios list above. It is OK to skip some cases if it is limited by K3D

  • Test the deployment on local Windows or Linux environment

  • Create documentation including github project for the project

By the mid term review we would expect you to have:

  • Simple mini HPCC Systems Deployment for ECL training

  • Logging or ECL VSCode Notebook support

 

Mentor

Xiaoming Wang
xiaoming.wang@lexisnexis.com

Backup Mentor: 

Godji Fortil
godson.fortil@lexisnexis.com

Skills needed

  • General knowledge of Linux, Docker and Kubernetes

  • Self-motivated to learn about HPCC Systems, ECL, HELM, Docker Desktop, K3D, Azure etc (guidance will be provided)

  • Willing to explore new technologies

  • Basic programming skills such as Unix shell (bash), Python, etc

Deliverables

Midterm

A github project instruction of setup K3D and some basic HPCC configuration

End of project

A complete github project with HPCC Cloud scenarios including documentation. A presentation in PPT or similar format.

Other resources

All pages in this wiki are subject to our site usage guidelines.