Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Add AWS EFS support to helm charts

Description

AWS doesn't has a out of box storageClass for network shared filesystem like azurefile (samba). Although using storageClass "gp2" (EBS) we can deploy HPCC cluster on AWS EKS but it can't scale due to no dynamic volume support for "Deployment" deployment type.

To has a similar azurefiles network filesystem storageclass we can add Elastic File Systems (fast NFS on AWS) support. This will include:
1) Setup a EFS
2) Add a EFS subchart under hpcc/charts/
3) Add a dependency in hpcc/Chart.yaml
4) Modify values.schema.json values.yaml for EFS entries
5) Modify templates/dali.yaml, datastorage-pvc.yaml anddllserver-pvc.yaml for EFS

Conclusion

None

Activity

Show:

Greg Panagiotatos April 23, 2020 at 12:44 PM

: See my proposed changes below:

Check the EFS region ensure it is configured for :
region: us-east-1

Keep in mind "gp2" on AWS cannot dynamically create volumes of "Deployment" type.
Pod scale will not work with "gp2" since HPCCSystems pods use "Deployment" type controller
for Pod replicas.
The "efs" storageClass is available if EFS is configured. It doesn't need dynamic volume and
Pod scale will work with "Deployment".
storageClass: ""

Above sentences will be appended to following:
If defined, storageClassName: <storageClass>
If set to "-", storageClassName: "", which disables dynamic provisioning
If undefined (the default) or set to null, no storageClassName spec is
set, choosing the default provisioner. (gp2 on AWS, standard on GKE, AWS & OpenStack)

Xiaoming Wang April 22, 2020 at 4:01 PM
Edited

can you help me rephrase following sentences which will in dockerfiles/hpcc/values.yaml?

Replace region if the EFS is configured in the other region
region: us-east-1

Keep in mind "gp2" on AWS cannot dynamicly creating volumes for "Deployment" type,
so Pod scale will not work with "gp2" since HPCCSystems pods use "Deployment" controller
for Pod replicas..
"efs" storageClass is available if EFS is configured which doesn't need dynamic volume and
Pod scale will work with "Deployment".
storageClass: ""

Above sentences will be appended to following:
If defined, storageClassName: <storageClass>
If set to "-", storageClassName: "", which disables dynamic provisioning
If undefined (the default) or set to null, no storageClassName spec is
set, choosing the default provisioner. (gp2 on AWS, standard on
GKE, AWS & OpenStack)

Fixed
Pinned fields
Click on the next to a field label to start pinning.

Details

Components

Assignee

Reporter

Priority

Fix versions

Pull Request URL

https://github.com/hpcc-systems/HPCC-Platform/pull/13679

Created April 22, 2020 at 1:19 PM
Updated June 29, 2020 at 2:26 PM
Resolved June 29, 2020 at 2:26 PM