Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

This project was completed by a student accepted on to the 2021 HPCC Systems Intern Program

Student work experience opportunities also exist for students who want to suggest their own project idea. Project suggestions must be relevant to HPCC Systems and of benefit to our open source community. 

Find out about the HPCC Systems Summer Internship Program.

The project proposal application period for 2020 summer internships is now closed. Check back in the Fall for details about applying to join our 2021 program.

Project Description

Project Description

Current HPCC System Kubernetes cluster deployment is mainly through helm install or generated a yaml file from helm template and deployed with kubectl command. As HPCC System cloud evolve more components probably will added such as monitoring, logging, third party software, etc. GitOps can be used to enhance helm solution and add automated update and version control features. Using GitOps to deploy and manage HPCC Systems Cloud probably will be a standard way for production, team development and testing.More information coming soon 

This project will explore some hot GitOps such as Flux, Argo CD, etc to deploy HPCC Systems cluster on public cloud but mainly will target Azure.


If you are interested in this project, please contact Contact Details

Completion of this project involves:

...

  • Deploy HPCC Cluster with Flux including Helm Controller. 
  • Handle additional HPCC cluster components such monitoring, logging or other third party components.
  • Explore various dynamic update for HPCC cluster through Flux
  • Rollback configuration with version control
  • Prefer using python for the code so don't need maintain both Unix shell and Windows Batch/PowerShell scripts
  • Complete github project with proper documentation.

By the mid term review we would expect you to have:

  • Coming soon
Mentor

Xiaoming Wang

Contact Details

xiaoming.wang@lexisnexis.com

Backup Mentor: Godson Fortil

Contact Details

fortil.godson@lexisnexis.com

Skills needed
  • General Cloud Environment knowledge: Docker, Kubernetes
AWS EC2, Client API (shell), S3, Docker, Jenkins, Packer
  • Azure and AWS
  • Unix Shell,
Pythoin
  • Python
  • Ability to build and test the HPCC system (guidance will be provided).
  • Ability to write test code. Knowledge of ECL is not a requirement since it should be possible to re-use existing code with minimal changes for this purpose. Links are provided below to our ECL training documentation and online courses should you wish to become familiar with the ECL  language.
Deliverables

Midterm

A github project to deploy HPCC cluster on local and Azure. Basic policy update and rollback functions

End of project

A complete github project with expected code and documentation.

Other resources
JIRA issue for this project: https://track.hpccsystems.com/browse/HPCC-24874
HPCC Systems Build Server Provision: https://github.com/xwang2713/cloud-image-build/tree/master/packer/aws