Some notes from meeting discussion on 6th June 2019:

Filter supplied to the remote  dafilesrv is on the actual file format, not on the expected file format.  That means that the client needs to always create the translators (so the filter can be mapped), and the remote dafilesrv code never needs to know the expected format.

Examples of filters that cannot be translated from expected to actual:  filter on unsigned2 field, which is an unsigned4 in the actual.  String fields that have changed size can be mapped to substring filters.  Filters on fields that have been introduced can be constant folded to either true or false.

Csv and xml with filters could

i) filter directly on the source csv

ii) map from csv to actual, filter and then project to projected.

If the filter cannot be translated from expected to actual, then the remote read needs to read the expected format, and the local code perform the filter and map from expected to the projected format.

It would be worth optimizing the case if the projected size is > than the actual size, and perform the translation on the client instead of remotely.  There would need to be a function in the translator to determine if it gets smaller/larger/unknown/stays the same/?  Could it still be worth projecting remotely if it avoids extra processing overhead on the client?