Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Add BEST attribute to DEDUP

Description

Add an attribute to DEDUP which allows you to indicate which record should be retained - rather than retaining the first or last.

It should avoid the need to sort the input dataset in some situations.

DEDUP(myDataset, a, b, BEST({c}))

Conclusion

None
100% Done
Type
Key
Summary
Priority
Story Points
Assignee
Status

relates to

Activity

Show:

Gavin Halliday August 9, 2016 at 9:44 AM

this is the issue I mentioned that you could look at if you run out of other things to do.

Gavin Halliday August 9, 2016 at 9:43 AM

Syntax in the grammar should be

BEST '(' sortList ')'

It requires the following changes:

  • Add some tests cases for the syntax - ascending and descending comparisons. Also add error cases to ecl/regress when it is combined with RIGHT (and other options that we do not support e.g., KEEP?).

  • Implement the syntax in hqlgram.y

  • Add a new flag and into IHThorDedupArg. Currently there is no getFlags() function. I would consider adding one and using DFcompareall, DFkeepledt, DFkeepbest, and reimplementing compareAll(), keepLeft() as inline functions (increase the version number)

  • Need to add a ICompare * queryCompareBest() member function - defaulting to null in the base class in eclhelper_base.hpp

  • Possibly check new option in canProcessInline() and other places that might be affected.

  • Generate the new flag and generate the compare function when activity is generated.

  • Add code to roxie to test and implement the new flag

  • Add code to Thor to test and implement the new flag

  • Add code to hthor to test and implement the new flag

  • Add documentation

  • Email SALT/KEL teams to warn them of the new feature.

Fixed
Pinned fields
Click on the next to a field label to start pinning.

Details

Components

Assignee

Reporter

Priority

Compatibility

Minor

Fix versions

Labels

Pull Request URL

https://github.com/hpcc-systems/HPCC-Platform/pull/9023

Created July 14, 2016 at 1:23 PM
Updated September 18, 2019 at 3:30 PM
Resolved December 1, 2016 at 4:43 PM

Flag notifications