Convert Automated Test Systems from Python2 to Python3

This project is available as a student work experience opportunity with HPCC Systems. Curious about other projects we are offering? Take a look at our Ideas List.

Student work experience opportunities also exist for students who want to suggest their own project idea. Project suggestions must be relevant to HPCC Systems and of benefit to our open source community. 

Find out about the HPCC Systems Summer Internship Program.

Project Description

The HPCC Systems Platform team has an automated test systems called OBT – Overnight Build and Test systems and the Smoketest (see repos listed in the table below). Both were created more than 7 years ago and used Python 2.7 which version has not been supported since the beginning of 2022. Therefore, we need to convert all of them. A large part of this project is related to Smoketest where the scheduler and the worker are written in Python. In OBT there are some log processing, reporting tools are in Python

There is a tool called 2to3 which can do some conversion, but it is unable to handle all our implementations, so there needs to be some understanding and manual fixes too. Moreover, there are some duplicated functionalities implemented in those systems what can be replaced by a common version. There are some dead (commented out, never called functions) and/or outdated code (like old base branch version related, e.g.: 6.x or 7.x) that can be removed. If a Python code is not unit testing ready, then at least it needs to be documented and if possible, change it. Our code needs some cleaning as well.

Completion of this project involves:

There are two kinds of deliverables:

• Code, provided in GitHub Pull Requests from cloned repositories. (The original codes are executed regularly to support our development work, so they are untouchable directly by this project.)
• Documentation

The expected deliverables are:

• All our Python code should run smoothly on the latest version.
• Clean up the code, remove all dead and/or outdated parts.
• Discover duplicated functionality, compile a list of them and give suggestions and code to common them.
• Create a list of all Python code which is not unit test enabled and if possible, make changes to it.

Bonus (if there is some time remaining on the project):

• Create some Python unit tests to test functionality.