Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 12 Next »

The proposal application period for the 2021 HPCC Systems intern Program is now open.

The deadline date for proposal applications is Friday 19th March 2021.

Discuss your ideas with the project mentor and send your final proposal to Lorraine Chapman.

Student work experience opportunities also exist for students who want to suggest their own project idea. Project suggestions must be relevant to HPCC Systems and of benefit to our open source community. 

Find out about the HPCC Systems Summer Internship Program.

Project Description

An increasing number of companies are moving to cloud environments. However, taking full advantage of the cloud infrastructure, particularly to get the job done while minimizing cost, remains a challenge. This project will try to explore how to use AWS EC2 Spot Instances  for some of our HPCC Systems development works, such as nightly builds, some function tests, etc. Spot Instances are much cheaper than their on-demand counterparts.  However, most spot instances will not have the pre-requisites suitable for HPCC Systems Development, such as the compute power. Thus, we may need to provision the spot instance before it can be used for our development purposes. As another option, we can create a Docker image with all of the prerequisite packages.  This project requires knowledge of AWS (EC2,  Client API, S3, etc),  Docker, Unix Shell, Jenkins, maybe Python and also HPCC Systems knowledge.  The students may not have all these skills but should be eager to learn the required tools or technologies.

If you are interested in this project, please contact Gordon Smith. 

Completion of this project involves:

  • Enable spot instances for  our nightly build on  at least one supported Linux distro, such as Ubuntu 18.04 or CentOS 7. It will be on AWS.
  • Use Jenkins and AWS Spot Instances  for some of HPCC Systems Development work, at least nightly build
  • Documentation
  • Explore the potential spot instance usage for HPCC Systems related jobs
  • Explore the similar on Microsoft AZure

By the mid term review we would expect you to have:

  • Enabled spot instances for  our nightly build on  at least one supported Linux distro, such as Ubuntu 18.04 or CentOS 7. It will be on AWS.
Mentor

Xiaoming Wang
Contact Details

Backup Mentor: Godson Fortil
Contact Details

Skills needed
  • General Cloud Environment knowledge
  • AWS EC2, Client API (shell), S3, Docker, Jenkins, Packer
  • Unix Shell, Pythoin
  • Ability to build and test the HPCC system (guidance will be provided).
  • Ability to write test code. Knowledge of ECL is not a requirement since it should be possible to re-use existing code with minimal changes for this purpose. Links are provided below to our ECL training documentation and online courses should you wish to become familiar with the ECL  language.
Deliverables

Midterm

  • Enable spot instances for  our nightly build on  at least one supported Linux distro, such as Ubuntu 18.04 or CentOS 7. It will be on AWS.

End of project

  • Use Jenkins and AWS Spot Instances  for some of HPCC Systems Development work, at least nightly build
  • Documentation
  • Explore the potential spot instance usage for HPCC Systems related jobs
  • Explore the similar on Microsoft AZure
Other resources
  • No labels

All pages in this wiki are subject to our site usage guidelines.