<p>I'm using bitnami/etcd chart and it has ability to create snapshots via <a href="https://github.com/bitnami/charts/blob/7faf745d6c2c3d81e9ac52db3c8de5418e1634b7/bitnami/etcd/templates/cronjob.yaml#L106" rel="nofollow noreferrer">EFS mounted pvc</a>.</p>
<p>However I get permission error after aws-efs-csi-driver is provisioned and PVC mounted to any <strong>non-root pod (user/gid is 1001)</strong></p>
<p>I'm using helm chart <a href="https://kubernetes-sigs.github.io/aws-efs-csi-driver/" rel="nofollow noreferrer">https://kubernetes-sigs.github.io/aws-efs-csi-driver/</a> version 2.2.0</p>
<p>values of the chart:</p>
<pre><code># you can obtain the fileSystemId with
# aws efs describe-file-systems --query &quot;FileSystems[*].FileSystemId&quot;
storageClasses:
  - name: efs
    parameters:
      fileSystemId: fs-exxxxxxx
      directoryPerms: &quot;777&quot;
      gidRangeStart: &quot;1000&quot;
      gidRangeEnd: &quot;2000&quot;
      basePath: &quot;/snapshots&quot;

# enable it after the following issue is resolved
# https://github.com/bitnami/charts/issues/7769
# node:
#   nodeSelector:
#     etcd: &quot;true&quot;
</code></pre>
<p>I then manually created the PV</p>
<pre><code>apiVersion: v1
kind: PersistentVolume
metadata:
  name: etcd-snapshotter-pv
  annotations:
    argocd.argoproj.io/sync-wave: &quot;60&quot;
spec:
  capacity:
    storage: 32Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  storageClassName: efs
  csi:
    driver: efs.csi.aws.com
    volumeHandle: fs-exxxxxxx

</code></pre>
<p>Then if I mount that EFS PVC in non-rood pod I get the following error</p>
<pre><code>➜ klo etcd-snapshotter-001-ph8w9                          
etcd 23:18:38.76 DEBUG ==&gt; Using endpoint etcd-snapshotter-001-ph8w9:2379
{&quot;level&quot;:&quot;warn&quot;,&quot;ts&quot;:1633994320.7789018,&quot;logger&quot;:&quot;client&quot;,&quot;caller&quot;:&quot;v3/retry_interceptor.go:62&quot;,&quot;msg&quot;:&quot;retrying of unary invoker failed&quot;,&quot;target&quot;:&quot;etcd-endpoints://0xc0005ea380/#initially=[etcd-snapshotter-001-ph8w9:2379]&quot;,&quot;attempt&quot;:0,&quot;error&quot;:&quot;rpc error: code = DeadlineExceeded desc = latest balancer error: last connection error: connection error: desc = \&quot;transport: Error while dialing dial tcp 10.120.2.206:2379: connect: connection refused\&quot;&quot;}
etcd-snapshotter-001-ph8w9:2379 is unhealthy: failed to commit proposal: context deadline exceeded
Error: unhealthy cluster
etcd 23:18:40.78 WARN  ==&gt; etcd endpoint etcd-snapshotter-001-ph8w9:2379 not healthy. Trying a different endpoint
etcd 23:18:40.78 DEBUG ==&gt; Using endpoint etcd-2.etcd-headless.etcd.svc.cluster.local:2379
etcd-2.etcd-headless.etcd.svc.cluster.local:2379 is healthy: successfully committed proposal: took = 1.6312ms
etcd 23:18:40.87 INFO  ==&gt; Snapshotting the keyspace
Error: could not open /snapshots/db-2021-10-11_23-18.part (open /snapshots/db-2021-10-11_23-18.part: permission denied)
</code></pre>
<p>As a result I have to spawn a new &quot;root&quot; pod, get inside the pod and manually adjust the permissions</p>
<pre><code>apiVersion: v1
kind: Pod
metadata:
  name: perm
spec:
  securityContext:
    runAsUser: 0
    runAsGroup: 0
    fsGroup: 0    
  containers:
  - name: app1
    image: busybox
    command: [&quot;/bin/sh&quot;]
    args: [&quot;-c&quot;, &quot;sleep 3000&quot;]
    volumeMounts:
    - name: persistent-storage
      mountPath: /snapshots
    securityContext:
      runAsUser: 0
      runAsGroup: 0
  volumes:
  - name: persistent-storage
    persistentVolumeClaim:
      claimName: etcd-snapshotter
  nodeSelector:
    etcd: &quot;true&quot;
</code></pre>
<pre><code>k apply -f setup.yaml
k exec -ti perm -- ash
cd /snapshots
/snapshots # chown -R 1001.1001 .
/snapshots # chmod -R 777 .
/snapshots # exit

➜ k create job --from=cronjob/etcd-snapshotter etcd-snapshotter-001
job.batch/etcd-snapshotter-001 created

➜ klo etcd-snapshotter-001-bmv79                          
etcd 23:31:10.22 DEBUG ==&gt; Using endpoint etcd-1.etcd-headless.etcd.svc.cluster.local:2379
etcd-1.etcd-headless.etcd.svc.cluster.local:2379 is healthy: successfully committed proposal: took = 2.258532ms
etcd 23:31:10.32 INFO  ==&gt; Snapshotting the keyspace
{&quot;level&quot;:&quot;info&quot;,&quot;ts&quot;:1633995070.4244702,&quot;caller&quot;:&quot;snapshot/v3_snapshot.go:68&quot;,&quot;msg&quot;:&quot;created temporary db file&quot;,&quot;path&quot;:&quot;/snapshots/db-2021-10-11_23-31.part&quot;}
{&quot;level&quot;:&quot;info&quot;,&quot;ts&quot;:1633995070.4907935,&quot;logger&quot;:&quot;client&quot;,&quot;caller&quot;:&quot;v3/maintenance.go:211&quot;,&quot;msg&quot;:&quot;opened snapshot stream; downloading&quot;}
{&quot;level&quot;:&quot;info&quot;,&quot;ts&quot;:1633995070.4908395,&quot;caller&quot;:&quot;snapshot/v3_snapshot.go:76&quot;,&quot;msg&quot;:&quot;fetching snapshot&quot;,&quot;endpoint&quot;:&quot;etcd-1.etcd-headless.etcd.svc.cluster.local:2379&quot;}
{&quot;level&quot;:&quot;info&quot;,&quot;ts&quot;:1633995070.4965465,&quot;logger&quot;:&quot;client&quot;,&quot;caller&quot;:&quot;v3/maintenance.go:219&quot;,&quot;msg&quot;:&quot;completed snapshot read; closing&quot;}
{&quot;level&quot;:&quot;info&quot;,&quot;ts&quot;:1633995070.544217,&quot;caller&quot;:&quot;snapshot/v3_snapshot.go:91&quot;,&quot;msg&quot;:&quot;fetched snapshot&quot;,&quot;endpoint&quot;:&quot;etcd-1.etcd-headless.etcd.svc.cluster.local:2379&quot;,&quot;size&quot;:&quot;320 kB&quot;,&quot;took&quot;:&quot;now&quot;}
{&quot;level&quot;:&quot;info&quot;,&quot;ts&quot;:1633995070.5507936,&quot;caller&quot;:&quot;snapshot/v3_snapshot.go:100&quot;,&quot;msg&quot;:&quot;saved&quot;,&quot;path&quot;:&quot;/snapshots/db-2021-10-11_23-31&quot;}
Snapshot saved at /snapshots/db-2021-10-11_23-31

➜ k exec -ti perm -- ls -la /snapshots                             
total 924
drwxrwxrwx    2 1001     1001          6144 Oct 11 23:31 .
drwxr-xr-x    1 root     root            46 Oct 11 23:25 ..
-rw-------    1 1001     root        319520 Oct 11 23:31 db-2021-10-11_23-31
</code></pre>
<h3>Is there a way to automate this?</h3>
<p>I have this setting in storage class</p>
<pre><code>gidRangeStart: &quot;1000&quot;
gidRangeEnd: &quot;2000&quot;
</code></pre>
<p>but it has no effect.</p>
<p>PVC is defined as:</p>
<pre><code>➜ kg pvc etcd-snapshotter -o yaml                
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  annotations:
    pv.kubernetes.io/bind-completed: &quot;yes&quot;
    pv.kubernetes.io/bound-by-controller: &quot;yes&quot;
    volume.beta.kubernetes.io/storage-provisioner: efs.csi.aws.com
  name: etcd-snapshotter
  namespace: etcd
spec:
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 32Gi
  storageClassName: efs
  volumeMode: Filesystem
  volumeName: etcd-snapshotter-pv
</code></pre>

DmitrySemenov
<p>By default the StorageClass field <code>provisioningMode</code> is unset, please set it to <code>provisioningMode: &quot;efs-ap&quot;</code> to enable dynamic provision with access point.</p>
