rollup1.ecl regression fails with 2 thor slaves

Description

Conclusion

None

Activity

Show:

Jacob Cobbett-Smith January 10, 2017 at 10:52 AM

Right, it's not because of the non-determistic DISTRIBUTE, because all rows originate on slave 1 as it's defined (with the inline dataset)...

It's because the query itself can have different results depending on the number of slaves.
In particular 2 slaves will always give 112839, any other number of slaves will give 11389.

The distribute by "r / (thorlib.nodes()+1)" , on a 2 slave system is always going to place the 1's,2 & 8 on the 1st node and 3,4,9 on the 2nd.
If 1 slave they'll all be on same slave so no issue, if >=3 slaves, values <=slaves will be on 1st node and others will either be grouped on higher slaves, or sequenced over higher slaves.
There would be the similar problem if there was an r value of 12 and slaves was 3.

I'll fix the test so consistent.

Mark Kelly January 9, 2017 at 8:26 PM

Two thor slaves

Jacob Cobbett-Smith January 9, 2017 at 3:14 PM

Haven't seen before, my guess is that both results are 'correct', but the result is non deterministic.
Result 3 - is a global rollup with a DISTRIBUTE in front of it, which means the rows will arrive at the rollup in a non-deterministic order.

How many slaves were in this setup?

Fixed
Pinned fields
Click on the next to a field label to start pinning.

Details

Components

Assignee

Reporter

Priority

Compatibility

Point

Fix versions

Pull Request URL

Created January 4, 2017 at 3:09 PM
Updated January 11, 2017 at 7:37 AM
Resolved January 11, 2017 at 7:37 AM