I couldn't sleep last night and wound up spending a great deal of time thinking about this issue.

HPCC is, among other things, a complete build system (compiler, linker, et al). If the runtime uses a particular set of libraries then we must offer those same libraries to code compiled by our toolchain in order to preserve version compatibility.

I think Gordon is right: All headers for all locally-installed libraries need to be installed, preferably located within a single directory (a local include/ if you will), and that directory needs to be searched first during compile phases. Library headers need to be installed whether the platform is built for debugging or not.

EDIT: The same is true for the libraries themselves (search our own libs/ directory first) but that is probably the case already. The reason that I assert that all of our install libraries are included is because we cannot predict what embedded code will attempt to link against.

Assuming the above is Made Real, does it matter what kind of #include is used in embedded C++? Will #include <foo.h> work the same as #include "foo.h" as far as our local headers go?

My initial thought (when Dan opened this issue) was to install all the headers for the third_party libs that are included in the release - see Install headers for third-party libs - HPCC (hpccsystems.com) 

We could start with ICU (I have a nagging feeling I won't be able to auto discover them, the same way I auto discover the libs...)

It worked before because the platform was using the version of icu that was installed within the operating system.  That means the header files would match the library files.  I think what is happening is that it is picking up the system icu header files (version 66), but linking to version 71 (since that is what the platform is using).

With the new build approach we will have a problem if there is embedded code that calls a shared library that the platform also uses, the versions do not match and the library cares about the mis-match. 

We could special case icu and include the header files so they are picked up instead.  This would also be a problem for plugins compiled independently of the platform.  

Any ideas??

I eventually realized that my own code was causing this. I have an embedded C++ function (CreateStringDeletionNeighborhood) that is including a unicode header. I will include the ZAP here for reference.

This was working under previous versions of 8.12.x. Was that accidental? Is there something I need to include in this code to make it work?

What I'm interested in is which of our files are causing the icu headers to be included...