title: docker container process running as non-root user cannot write to docker volume
body: <h2>TLDR</h2>
  <ul>
  <li>everyone recommends processes within containers should never run as root</li>
  <li>(except for <code>kubernetes</code>) there doesn't seem to be a good devops/configuration-as-code approach to get the right owner/permissions set on docker volumes, so the (non-root) user cannot write to the volume</li>
  </ul>
  <p>What is then good practice when running a container process as a <code>non-root</code> user and I want to write to a (cloudstor, aws-ebs) docker volume.</p>
  <hr />
  <h2>Long story</h2>
  <p>It is considered bad practice in (and outside) docker containers to run processes as root (see for example <a href="https://medium.com/@mccode/processes-in-containers-should-not-run-as-root-2feae3f0df3b#:%7E:text=Most%20containerized%20processes%20are%20application,easy%20method%20to%20limit%20access." rel="noreferrer">ref1</a>, <a href="https://opensource.com/article/18/3/just-say-no-root-containers" rel="noreferrer">ref2</a>, ...). This might have security implications.</p>
  <p>But when we start using docker volumes and that non-root user tries to write to the volume, the trouble begins. I fail to find a clean solution that will work on cloud infrastructure, without manual intervention. The working solutions I found all seem to fall short on some point (security, maintainability, ...).</p>
  <p>As a side-note, we are deploying on <code>docker-swarm</code> using <code>cloudstor</code> to provision <code>aws-ebs</code> volumes. We hope to move to kubernetes one day, but we don't have kubernetes yet, so we try to find an alternative solution for our setup.</p>
  <h2>Solutions / Workarounds considered</h2>
  <h3>1. Pre-create the volume within the docker image</h3>
  <p>As proposed <a href="https://github.com/docker/compose/issues/3270" rel="noreferrer">here</a>, if <code>docker-compose</code> creates a <strong>new</strong> volume, the permissions on the directory inside the image will be propagated.</p>
  <p><strong>downsides:</strong></p>
  <ul>
  <li>this won't work if the volume existed before, or if it is a folder on disk</li>
  <li>if the volume is provisioned with <code>cloudstor</code>, probably this won't work either, because it won't be <code>docker-compose</code> provisioning the volume (not tested)</li>
  </ul>
  <h3>2. Use volumes-provisioner</h3>
  <p>hasnat created a <a href="https://hub.docker.com/r/hasnat/volumes-provisioner" rel="noreferrer">volumes-provisioner</a> image which could set the correct folder permissions just before the real container starts.</p>
  <p><strong>downsides:</strong></p>
  <ul>
  <li>need to add an extra service in the docker stack. This service dies almost instantly (after setting permissions).</li>
  <li>The real container needs to <em>depends_on</em> the volumes_provisioner. When redeploying the same stack (after configuration changes), the order of execution is not guaranteed</li>
  <li><code>ebs</code> volumes can only be mounted on a single docker container, which caused many deployment issues</li>
  </ul>
  <h3>3. Use <code>docker run</code> to correct file permissions</h3>
  <p>Once the real container is running with the volume mounted (but still with the wrong permissions), we call</p>
  <pre><code>docker run --rm -u root -v ${MOUNT}:${TARGET} { real_image } chown -R user:group ${TARGET}
  </code></pre>
  <p><strong>downsides:</strong></p>
  <ul>
  <li>the <code>ebs</code> volume can only be mounted in one container, so this will create conflicts</li>
  <li>this command can only be run after the docker-stack has been deployed (otherwise the volume hasn't been provisioned yet), so there will be a delay between the real container startup and the correct permissions. This means that at startup, the real container finds the volume with the wrong permissions, so this will only work if the service keeps checking if the permissions have been corrected.</li>
  </ul>
  <h3>4. Change ownership when starting the container</h3>
  <p>This implies:</p>
  <ul>
  <li>starting the process as <code>root</code> user (otherwise we don't have the right to change directory owner/permissions)</li>
  <li>changing ownership/permissions</li>
  <li>switching to <code>non-root</code> user</li>
  </ul>
  <p><strong>downsides:</strong></p>
  <ul>
  <li>there is still a (minor?) time period when the container process is running as root (security implication?)</li>
  <li>need to hack entrypoints, override user,... of official images to get this working</li>
  </ul>
  <h3>5. Just run as root</h3>
  <p>This is the easiest solution, but then what about security? And everybody recommending <strong>not</strong> to do this?</p>
  <h3>6. Use kubernetes</h3>
  <p>As suggested <a href="https://stackoverflow.com/a/46769504/2082964">here</a>, with kubernetes we can assign a group id to a volume. This seems confirmed in the <a href="https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#configure-volume-permission-and-ownership-change-policy-for-pods" rel="noreferrer">kubernetes documentation for pods</a>.</p>
  <p><strong>downsides:</strong></p>
  <ul>
  <li>(sadly) we don't use <code>kubernetes</code> yet.</li>
  <li>(not tested.)</li>
  </ul>
  <h3>7. create folders on the file system with correct permissions</h3>
  <p>Make sure directories exist on the file system with the correct owner/permissions.</p>
  <p><strong>downsides</strong></p>
  <ul>
  <li>this is not cloud-storage... what if the container switches to another node? or if the server crashes? (which is why we use <code>cloudstor</code>, which allows us even to switch availability zones)</li>
  <li>doesn't seem very &quot;configuration-as-code&quot;</li>
  </ul>