HPCC Systems 7.6.x Releases

Welcome to the Red Book for HPCC Systems^® 7.6.x series. There are several sections in this Red Book as follows:

• General HPCC Systems core platform 

• Significant new features core platform

• ECL IDE

Users may benefit from glancing at other Red Book entries when making a large jump between releases.

Here's how to contact us if you find an issue or want to add something to the Red Book:

• To raise an issue, use our Community Issue Tracker. Please create yourself an account if you don't already have one, to get automatic updates as your issue progresses through the workflow.

• To ask a developer a technical question about something you are doing or have encountered, post in the Developer Forum.

General HPCC Systems core platform

New - Encountering an issue where embedded Java fails

The platform does not currently always enforce a Java Development Kit (JDK) dependency and the embedded Java plugin (javaembed) can fail if a JDK is not available. If you encounter this error, install the appropriate JDK for your Ubuntu distro or the openjdk-devel package for Centos. 

https://track.hpccsystems.com/browse/HPCC-25879

Keyed Join fix for the remote handling of parts from othr clusters

When a smaller Thor cluster read a file from a larger Thor where the larger Thor file overlapped some slaves of the smaller cluster, it could find non-primary copies of the file parts local to the slaves. The effect of this was that other slaves working on KJ would send requests to those local slaves, but did not map to the correct (non-primary) copy, resulting in a 'Failed to open' file error.

A workaround is to add this option:

^{#option('remoteKeyedLookup', false);}

which forces all KJ part thread handlers to access the keyed parts directly.

https://track.hpccsystems.com/browse/HPCC-23163

Thor jobs locking up when using SMART join

This may happen in specific circumstances, when using a SMART join that has failed over to a local hash join and the LHS is stopped early. The workaround if this issue is being hit in builds prior to this fix (7.4.34), would be to switch from using a SMART join, to either a regular HASH join, or a standard JOIN.

Note:  Because this only happens when it has already run out of memory (and failed over to a local hash join), coding specifically as a HASH join is no worse performance wise.

https://track.hpccsystems.com/browse/HPCC-23068

Log file format change

The default log file format has changed in 7.6.0 to add an Audience field. This is configurable in the environment.conf file.  
Any existing tools you use for parsing or extracting information from log files may need to be modified. 

The new standard message format is:

Python 3 is now the default

Beginning in 7.6.0, the platform supports Python3 by default. It still includes plugins for both Python2 and
Python3, but only one may be safely enabled at a time as the Python libraries export the same symbols for both versions.
Enabling both may lead to unpredictable results including segmentation faults or undefined symbol errors.

By default, the Python2 plugin is present but disabled, and the Python3 plugin is present and enabled. If you
want to use Python2 modify your environment.conf file according to the example in the file. A cluster updated from
an older version may not have the appropriate entries in the environment.conf file to support either version of Python.

Fix to HASH functions changes behavior

Prior to 7.6.0, the HASH32 and HASH64 operations on a VARSTRING did not ignore trailing spaces as they should have. 

Beginning in version 7.6.0, trailing spaces ARE ignored.

For versions prior to 7.6.0, you can use TRIM to remove trailing spaces. For example:

^{HASH32(TRIM(myVarstring));
HASH64(TRIM(myVarstring));}