Allow platform builds to employ envmod to make changes to the environment.xml when required

Description

Ops ran into a problem after upgrading roxies to OSS 8.0.24-1 where query compiles were not working. The users were getting error:

Exception(s):

20010: 2021-07-06 14:40:55 GMT: Resource DeployWorkunitsAccess : WsWorkunits::WUDeployWorkunit: Permission denied

Working with Kevin Wang, Ops was given instructions to manually edit the environment.xml file in multiple places to insert a new feature to correct the error.

This is NEVER a good idea. We need a way for the platform to update the existing environment.xml file, as needed, when new features need to be included as part of a build deployment. 

Conclusion

None

Activity

Show:

James Wiltshire April 24, 2023 at 10:13 PM

I noted in one of these comments:

"This tool is being obsoleted as we move away from non-container installations."

Is this, in fact, the direction of HPCC?  To no longer support non-container installations?

And, is configmgr no longer supported?

 

Anthony Fishbeck July 13, 2021 at 2:02 PM

  Ok, so, the solution I described is the real fix for the problem, but it's a chunk of work in bare metal and we aren't really focused on major improvements to bare metal right now.   should probably go ahead with his suggestion if it gives operations what they need.

 

Stuart Ort July 13, 2021 at 12:44 PM

@Kevin - if you go with Tony's suggestion, make sure the file is part of the installation.  It will also need to be merged into the version of the platform where this feature was first introduced and users will have to upgrade to the latest point release.  There will need to be testing done since we don't normally have bare metal components read multiple configuration files.  This implementation will mean that there are no configmgr changes needed and this Jira should be assigned to you

Anthony Fishbeck July 12, 2021 at 7:34 PM

 the default settings can be loaded from "opt/HPCCSystems/componentfiles/applications/*/ldap_authorization_map.yaml"... that's how I do it in the cloud.  Then we can be sure everything stays in sync.

The code for loading these are in application_config.cpp, but that loads a lot of stuff, we should make sure this security tags part is in a shared function.

So just like in the cloud... 1. load defaults from application yaml files.  2. merge user settings. 

Then you end up with defaults for anything the user doesn't provide.

(In this case "the user" is environment.xml/esp.xml)

Note that the files are specific for the application for example eclwatch, or eclqueries (ws_ecl).  The files are also named based on the security manager.  So we just need to load the right ones based on those two things.

Ken Rowland July 12, 2021 at 7:00 PM

A modification template will be written that adds the sections outlined by  in a previous comment. The template will be made available so that the environment can be updates. It will be the responsibility of the DevOps team to run the envmod command line utility using the modification template to update any required environment files. The changed environment file will need to be pushed to each node in the cluster.

 

Fixed
Pinned fields
Click on the next to a field label to start pinning.

Details

Assignee

Reporter

Priority

Compatibility

Point

Fix versions

Roadmap

Not applicable

Created July 8, 2021 at 1:19 AM
Updated April 24, 2023 at 10:13 PM
Resolved October 7, 2021 at 10:04 AM

Flag notifications