Convert Automated Test Systems from Python2 to Python3

This project is available as a student work experience opportunity with HPCC Systems. Curious about other projects we are offering? Take a look at our Ideas List.

Student work experience opportunities also exist for students who want to suggest their own project idea. Project suggestions must be relevant to HPCC Systems and of benefit to our open source community. 

Find out about the HPCC Systems Summer Internship Program.

Project Description

The HPCC Systems Platform team has an automated test systems called OBT – Overnight Build and Test systems - and the Smoketest (see repos listed in the table below). Both were created more than 7 years ago and used Python 2.7 whose version is no longer supported since the beginning of 2022. Therefore, we need to upgrade the python version in both of them. A large part of this project is related to Smoketest where the scheduler and the worker are written in Python. In OBT, some log processing and reporting tools are written in Python as well.

There is a tool called 2to3 which can do some conversion, but it is unable to handle all our implementations, so there is a need for some investigation and manual fixes too. Moreover, there are some duplicated functionalities implemented in those systems that can be replaced by a common version. There are some dead (commented out, never called functions) and/or outdated code (like old base branch version related, e.g.: 6.x or 7.x) that can be removed as well. If a Python code is not unit testing ready, then at least it needs to be documented and if possible, changed. Our code needs some cleaning as well.

Completion of this project involves:

There are two kinds of deliverables:

• Code, provided in GitHub Pull Requests from cloned repositories. (The original codes are executed regularly to support our development work, so they are untouchable directly by this project.)
• Documentation

The expected deliverables are:

• All our Python code should run smoothly on the latest version.
• Clean up the code, remove all dead and/or outdated parts.
• Discover duplicated functionality, compile a list of them and give suggestions and code to commonize them.
• Create a list of all Python code which is not unit test enabled and if possible, make changes to it.

Bonus (if there is some time remaining on the project):

• Create some Python unit tests to test functionality.