<p>I am trying to host a web app in a container with read only file system. Whenever I try to configure the root file system as read only through the <code>SecurityContext</code> of the container I get the following error:</p>
<pre><code>    Ports:          80/TCP, 443/TCP
    Host Ports:     0/TCP, 0/TCP
    State:          Terminated
      Reason:       Error
      Exit Code:    137
      Started:      Thu, 23 Sep 2021 18:13:08 +0300
      Finished:     Thu, 23 Sep 2021 18:13:08 +0300
    Ready:          False
</code></pre>
<p>I've tried to achieve the same using an AppArmor profile as follows:</p>
<pre><code>profile parser-profile flags=(attach_disconnected) {
  #include &lt;abstractions/base&gt;
   ...
  deny /** wl,
   ...
</code></pre>
<p>Unfortunately the result is the same.</p>
<p>What I assume is happening is that the container is not capable of saving the files for the web app and fails.</p>
<p>In my scenario, I will be running untrusted code and I must make sure that users are not allowed to access the file system.</p>
<p>Any ideas of what I am doing wrong and how can I achieve a read only file system?</p>
<p>I am using AKS and below is my deployment configuration:</p>
<pre><code>apiVersion: v1
kind: Service
metadata:
  name: parser-service
spec:
  selector:
    app: parser
  ports:
  - port: 80
    targetPort: 80
    protocol: TCP
    name: http
  - port: 443
    targetPort: 443
    protocol: TCP
    name: https
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: parser-deployment
spec:
  replicas: 5
  selector:
    matchLabels:
      app: parser
  template:
    metadata:
      labels:
        app: parser
      annotations:
        container.apparmor.security.beta.kubernetes.io/parser: localhost/parser-profile
    spec:
      containers:
      - name: parser
        image: parser.azurecr.io/parser:latest
        ports:
        - containerPort: 80
        - containerPort: 443
        resources:
          limits:
            cpu: &quot;1.20&quot;
        securityContext:
          readOnlyRootFilesystem: true


</code></pre>
<p>Edit: I also tried creating a cluster level PSP which also did not work.</p>

<p>I managed to replicate your issue and achieve read only filesystem with exception for one directory.</p>
<p>First, worth to note that you are using both solutions in your deployment - the AppArmor profile and SecurityContext. As AppArmor seems to be much more complex and needs configuration to be done per node I decided to use only SecurityContext as it is working fine.</p>
<p>I got this error that you mention in the comment:</p>
<pre><code>Failed to create CoreCLR, HRESULT: 0x80004005
</code></pre>
<p>This error doesn't say to much, but after some testing I found that it only occurs when you are running the pod which filesytem is read only -  the application tries to save files but cannot do so.</p>
<p>The app creates some files in the <code>/tmp</code> directory so the solution is to mount <code>/tmp</code> using <a href="https://kubernetes.io/docs/concepts/storage/volumes/" rel="noreferrer">Kubernetes Volumes</a>  so it will be read write. In my example I used <a href="https://kubernetes.io/docs/concepts/storage/volumes/#emptydir" rel="noreferrer">emptyDir</a> but you can use any other volume you want as long as it supports writing to it. The deployment configuration (you can see that I added <code>volumeMounts</code> and <code>volumes</code> and the bottom):</p>
<pre><code>apiVersion: v1
kind: Service
metadata:
  name: parser-service
spec:
  selector:
    app: parser
  ports:
  - port: 80
    targetPort: 80
    protocol: TCP
    name: http
  - port: 443
    targetPort: 443
    protocol: TCP
    name: https
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: parser-deployment
spec:
  replicas: 5
  selector:
    matchLabels:
      app: parser
  template:
    metadata:
      labels:
        app: parser
    spec:
      containers:
      - name: parser
        image: parser.azurecr.io/parser:latest
        ports:
        - containerPort: 80
        - containerPort: 443
        resources:
          limits:
            cpu: &quot;1.20&quot;
        securityContext:
          readOnlyRootFilesystem: true
        volumeMounts:
        - mountPath: /tmp
          name: temp
      volumes:
      - name: temp
        emptyDir: {}
</code></pre>
<p>After executing into pod I can see that pod file system is mounted as read only:</p>
<pre><code># ls   
app  bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var
# touch file
touch: cannot touch 'file': Read-only file system
# mount 
overlay on / type overlay (ro,...)
</code></pre>
<p>By running <code>kubectl describe pod {pod-name}</code> I can see that <code>/tmp</code> directory is mounted as read write and it is using <code>temp</code> volume:</p>
<pre><code>Mounts:
  /tmp from temp (rw)
</code></pre>
<p>Keep in mind if you are using other directories (for example to save files) you need to also mount them the same way as the <code>/tmp</code>.</p>
