Migrate and improve regression testing in GitHub Actions

This project was completed by a student accepted on to the 2024 HPCC Systems Intern Program.

Project Description

The HPCC Systems Platform team has an automated test system called OBT – Overnight Build and Test systems that is currently utilized to perform regression testing on ECL bundles, such as GNN and Machine Learning bundles, whenever a change to the HPCC Systems source code is made. However, more modern DevOps approaches suggest performing regression testing as part of automated workflows such as GitHub actions. GitHub Actions is a continuous integration and continuous delivery (CI/CD) platform that allows you to automate your build, test, and deployment pipeline. You can create workflows that build and test every pull request to your repository, or deploy merged pull requests to production. Therefore, the first goal of this project is to migrate the regression testing for ECL bundles from scripts contained within the OBT to GitHub Actions.

Also, the HPCC Systems user documentation and the HPCC Systems GitHub README files usually contain hyperlinks to the HPCC Systems website and other external sources that are constantly changing. This can result in the famous error 404 (i.e., page not found). While manual testing by a human can still be employed to verify and update the hyperlinks, this is not the most reliable way. Therefore, the second goal of this project is to develop an automated testing of hyperlinks in HPCC Systems user documents and GitHub README files using GitHub Actions.

To accomplish this, the following requirements must be considered:

• The solution should be implemented using GitHub Actions as the CI/CD framework.

• For the first goal, the solution should be capable of testing all the ML and supported bundles (see ECL bundle repository in the resources area below).

• For the second goal, the solution must be capable of automatically extracting all hyperlinks from the source files of the documents, either in XML or markdown format (a script will need to be developed to parse these files).

Completion of this project involves:

• Fully functional code according to the requirements above.

• Creation of documentation including a GitHub repository for the project.

• A blog, a recorded presentation, and a poster artifact about your project (see examples from previous years here).