<p>I'm running a cluster on AWS EKS. Container(StatefulSet POD) that currently running has Docker installation inside of it. </p>

<p>I ran this image as Kubernetes StatefulSet in my cluster. Here is my yaml file,</p>

<pre><code>apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: jenkins
  labels:
    run: jenkins
spec:
  serviceName: jenkins
  replicas: 1
  selector:
    matchLabels:
      run: jenkins
  template:
    metadata:
      labels:
        run: jenkins
    spec:
      securityContext:
        fsGroup: 1000
      containers:
      - name: jenkins
        image: 99*****.dkr.ecr.&lt;region&gt;.amazonaws.com/&lt;my_jenkins_image&gt;:0.0.3
        imagePullPolicy: Always
        ports:
        - containerPort: 8080
          name: jenkins-port
</code></pre>

<p>Inside this POD, I can not run any docker command which gives a ERROR:</p>

<blockquote>
  <p>/etc/init.d/docker: 96: ulimit: error setting limit (Operation not permitted)</p>
</blockquote>

<p>In my research, I went through some artcile which did not fix my issue.
I have listed down solution that i tried but not fixed in my case</p>

<p><strong>First solution: (I ran inside the container)</strong>
<a href="https://stackoverflow.com/questions/24318543/how-to-set-ulimit-file-descriptor-on-docker-container-the-image-tag-is-phusion">aricle link</a></p>

<pre><code>$ sudo service docker stop
$ sudo bash -c "echo \"limit nofile 262144 262144\" &gt;&gt; /etc/init/docker.conf"
$ sudo service docker start
</code></pre>

<p><strong>Second solution: (I ran inside the container)</strong></p>

<pre><code>ulimit -n 65536 in /etc/init.d/docker
</code></pre>

<p><strong>Third solution: ** <a href="https://unix.stackexchange.com/questions/450799/getting-operation-not-permitted-error-when-setting-ulimit-for-memlock-in-a-doc">article link</a>
This seems a far better answer, which i could not add into my configuration file.
it says, run pod with as privilaged. But there is no way to add that option in ***Kubernetes StatefulSet*</strong> .
So I tried by adding a SecurityContext <strong>(securityContext:fsGroup: 1000)</strong> like this inside configuration file,</p>

<pre><code>spec:
  serviceName: jenkins
  replicas: 1
  selector:
    matchLabels:
      run: jenkins
  template:
    metadata:
      labels:
        run: jenkins
    spec:
      securityContext:
        fsGroup: 1000
</code></pre>

<p>still it does not work.</p>

<p><strong>Note :same image worked on Docker swarm</strong></p>

<p>Anyhelp would be appreciated!</p>

<p>I had this issue with Elastic Search and adding <code>initContainer</code> worked. In this case it could be the solution: </p>

<pre><code>spec:
  .
  .
  .
  initContainers:  
  - name: increase-fd-ulimit
    image: busybox
    command: ["sh", "-c", "ulimit -n 65536"]
    securityContext:
      privileged: true
</code></pre>

<p>If it doesn't work, there is a <a href="https://github.com/kubernetes/kubernetes/issues/3595#issuecomment-487919341" rel="nofollow noreferrer">second</a> way to solve this problem which includes creating a new Dockerfile or changing existing:</p>

<pre><code>FROM 99*****.dkr.ecr.&lt;region&gt;.amazonaws.com/&lt;my_jenkins_image&gt;:0.0.3
RUN ulimit -n 65536
USER 1000
</code></pre>

<p>and change securityContext to:</p>

<pre><code>  securityContext:
    runAsNonRoot: true
    runAsUser: 1000
    capabilities:
      add: ["IPC_LOCK"]
</code></pre>
