<p>I was trying <code>elasticsearch</code> deployment on Kubernetes (EKS) cluster with persistence volume (EBS), refering to <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html" rel="nofollow noreferrer">https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html</a></p>

<pre><code>apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: elasticsearch
spec:
  replicas: 1
  template:
    metadata:
      labels:
        run: elasticsearch
    spec:
      initContainers:
        - name: init-sysctl
          image: busybox
          imagePullPolicy: IfNotPresent
          command: ["sysctl", "-w", "vm.max_map_count=262144"]
          securityContext:
            privileged: true
      containers:
      - name: elasticsearch
        image: elasticsearch:6.6.1
        imagePullPolicy: IfNotPresent
        securityContext:
          privileged: true
          capabilities:
            add:
              - IPC_LOCK
        ports:
        - containerPort: 9300
          protocol: TCP
        resources:
          limits:
            cpu: 1
        env:
        - name: CLUSTER_NAME
          value: elasticsearch
        - name: ES_JAVA_OPTS
          value: "-Xms256m -Xmx512m"
        volumeMounts:
          - name: elasticsearch-storage
            mountPath: /usr/share/elasticsearch/data
      volumes:
      - name: elasticsearch-storage
        persistentVolumeClaim:
          claimName: elasticsearch
</code></pre>

<p>But pod failed with </p>

<pre><code>OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[2019-03-13T18:03:08,459][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [unknown] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: Failed to create node environment
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.6.1.jar:6.6.1]
    at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.6.1.jar:6.6.1]
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) ~[elasticsearch-6.6.1.jar:6.6.1]
Caused by: java.lang.IllegalStateException: Failed to create node environment
    at org.elasticsearch.node.Node.&lt;init&gt;(Node.java:298) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.node.Node.&lt;init&gt;(Node.java:265) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.bootstrap.Bootstrap$5.&lt;init&gt;(Bootstrap.java:212) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-6.6.1.jar:6.6.1]
    ... 6 more
Caused by: java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data/nodes
    at sun.nio.fs.UnixException.translateToIOException(UnixException.java:90) ~[?:?]
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) ~[?:?]
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) ~[?:?]
    at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:385) ~[?:?]
    at java.nio.file.Files.createDirectory(Files.java:689) ~[?:?]
    at java.nio.file.Files.createAndCheckIsDirectory(Files.java:796) ~[?:?]
    at java.nio.file.Files.createDirectories(Files.java:782) ~[?:?]
    at org.elasticsearch.env.NodeEnvironment.lambda$new$0(NodeEnvironment.java:270) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.env.NodeEnvironment$NodeLock.&lt;init&gt;(NodeEnvironment.java:203) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.env.NodeEnvironment.&lt;init&gt;(NodeEnvironment.java:267) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.node.Node.&lt;init&gt;(Node.java:295) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.node.Node.&lt;init&gt;(Node.java:265) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.bootstrap.Bootstrap$5.&lt;init&gt;(Bootstrap.java:212) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-6.6.1.jar:6.6.1]
    ... 6 more
</code></pre>

<p>From this <a href="https://discuss.elastic.co/t/elastic-elasticsearch-docker-not-assigning-permissions-to-data-directory-on-run/65812/2" rel="nofollow noreferrer">discussion</a>, it looks like permission issue on <code>/usr/share/elasticsearch/data</code> dir. </p>

<p>following <code>initContainers</code> helped to get me around this issue. </p>

<pre><code>initContainers:
  - name: init-sysctl
    image: busybox
    imagePullPolicy: IfNotPresent
    #command: ["sysctl", "-w", "vm.max_map_count=262144"]
    command: ["/bin/sh","-c"]
    args: ["sysctl -w vm.max_map_count=262144; chown -R 1000:1000 /usr/share/elasticsearch/data"]
    securityContext:
      privileged: true
    volumeMounts:
      - name: elasticsearch-storage
        mountPath: /usr/share/elasticsearch/data
</code></pre>

<p>Whats is correct way to fix this issue ?</p>

roy
<p>The solution you are using is a workaround , but you can do it a bit kubernetes way is explained here:</p>

<p><a href="https://stackoverflow.com/questions/46351051/permission-denied-when-chown-on-elasticsearch-data-directory-in-kubernetes-state">Permission denied when chown on elasticsearch data directory in kubernetes statefulset</a></p>

<p>So in your case it should be like:</p>

<pre><code>....

spec:
  ...
  securityContext:
    fsGroup: 1000
</code></pre>
