Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Invalid filter serialization created (caught by dafilesrv), if substring filter on DATA

Description

One of the regression suite queries, indexsubstring2.ecl, causes dafilesrv to spot an invalid serialized filter.
It throws the exception [ "Invalid filter - missing closing '" ]

The issue is not seen, because remote looksup like this automatically failover to direct reading.

It looks like this is being caused by the filter serialization of a substring filter on a DATA field.

The ECL in the query that was being used at the time was:

INDEX({ data8 postcode }, { big_endian unsigned integer8 __filepos }, INTERNAL('gl81')); FILTER(mkKeyed(....postcode[..four] IN IF(four = 4, PartialPostcodeStored8, ALL)));

where the field 'postcode' is a DATA8.

Stepping through the serialization code, in this stack:

libjlib.so!appendStringExpandControl(StringBuffer & out, unsigned int len, const char * src, bool addBreak, bool isCpp, bool isUtf8) (/home/jsmith/git/HPCC-Platform/system/jlib/jstring.cpp:1624) libjlib.so!appendUtf8AsECL(StringBuffer & out, unsigned int len, const char * src) (/home/jsmith/git/HPCC-Platform/system/jlib/jstring.cpp:1653) libeclrtl.so!ValueTransition::describe(const ValueTransition * const this, const RtlTypeInfo & type, StringBuffer & out) (/home/jsmith/git/HPCC-Platform/rtl/eclrtl/rtlnewkey.cpp:461) libeclrtl.so!ValueSet::describe(const ValueSet * const this, StringBuffer & out) (/home/jsmith/git/HPCC-Platform/rtl/eclrtl/rtlnewkey.cpp:1183) libeclrtl.so!ValueSet::serialize(const ValueSet * const this, StringBuffer & out) (/home/jsmith/git/HPCC-Platform/rtl/eclrtl/rtlnewkey.cpp:1076) libeclrtl.so!SetFieldFilter::serialize(const SetFieldFilter * const this, StringBuffer & out) (/home/jsmith/git/HPCC-Platform/rtl/eclrtl/rtlnewkey.cpp:1536) libdafsclient.so!CRemoteFilteredFileIOBase::CRemoteFilteredFileIOBase(CRemoteFilteredFileIOBase * const this, SocketEndpoint & ep, const char * filename, IOutputMetaData * actual, IOutputMetaData * projected, const RowFilter & fieldFilters, unsigned long long chooseN) (/home/jsmith/git/HPCC-Platform/fs/dafsclient/rmtfile.cpp:2193)

it could be seen that appendStringExpandControl, was appending null terminator characters to the output for positions 4-8.

Conclusion

None

Activity

Show:
Fixed
Pinned fields
Click on the next to a field label to start pinning.

Details

Components

Assignee

Reporter

Priority

Compatibility

Point

Fix versions

Pull Request URL

https://github.com/hpcc-systems/HPCC-Platform/pull/14427
https://github.com/hpcc-systems/HPCC-Platform/pull/14427

Created December 17, 2020 at 9:51 AM
Updated January 5, 2021 at 9:20 AM
Resolved January 5, 2021 at 9:20 AM

Flag notifications