<p>I am running Airflow on Google's Cloud Composer. I am using the <a href="https://airflow.apache.org/_api/airflow/contrib/operators/kubernetes_pod_operator/index.html#airflow.contrib.operators.kubernetes_pod_operator.KubernetesPodOperator" rel="nofollow noreferrer">KubernetesPodOperator</a> and would like to mount a google storage bucket to a directory in pod via <a href="https://github.com/GoogleCloudPlatform/gcsfuse" rel="nofollow noreferrer">gcsfuse</a>. It seems like to do this I need to give k8s privileged security context as specified <a href="https://github.com/maciekrb/gcs-fuse-sample" rel="nofollow noreferrer">here</a>. It seems like airflow recently added the security_context parameter to the KubernetesPodOperator. The security context I am specifying in the operator is :</p>

<pre><code>security_context = {
  'securityContext': {
    'privileged': True,
    'capabilities':
      {'add': ['SYS_ADMIN']}
  }
}
</code></pre>

<p>When I try running <code>airflow test dag_id task_id date</code> in the airflow worker, the pod launches and when the code tries to mount the bucket via gcsfuse it throws the error <code>"fusermount: fuse device not found, try 'modprobe fuse' first"</code>. This makes it seems as the security_context is not working (<a href="https://serverfault.com/questions/924793/gcsfuse-in-google-app-engine-flexible-custom-runtime">ex.</a>).</p>

<p>Am I misunderstanding what the security_context parameter in the operator and/or is my securityContext dictionary definition invalid?</p>

Nithin Saripalli
<p>The <code>security_context</code> kwarg for the KubernetesPodOperator sets the security context for the pod, not a specific container within the pod, so it only supports the options outlined in <a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.19/#podsecuritycontext-v1-core" rel="nofollow noreferrer"><code>PodSecurityContext</code></a>. Since the parameters you are specifying aren't valid for a pod's security context, they are being discarded.</p>
<p>The <code>privileged</code> and <code>capabilities</code> properties are only valid as part of a container's <a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.19/#securitycontext-v1-core" rel="nofollow noreferrer"><code>SecurityContext</code></a>, meaning you'll need to somehow set them on the pod's <em><strong>container</strong></em> spec. You can do this by defining the entire pod spec yourself (as opposed to having the operator generate it for you). Using KubernetesPodOperator, you can set <code>full_pod_spec</code> or <code>pod_template_file</code> to specify a Kubernetes API Python object, or path to an object YAML, which would then be used to generate the pod. Example using the former:</p>
<pre><code>from airflow.contrib.operators.kubernetes_pod_operator import KubernetesPodOperator
import kubernetes.client.models as k8s

pod = k8s.V1Pod()
pod.spec = k8s.V1PodSpec()
pod.spec.containers = [
    k8s.V1Container(
      ...,
      security_context={
          'privileged': True,
          'capabilities': {'add': ['SYS_ADMIN']}
      }
    )
]

# Equivalent to setting security_context from the operator
pod.spec.security_context = {}

t1 = KubernetesPodOperator(..., full_pod_spec=pod)
</code></pre>
<p>If you want to use <code>pod_template_file</code> with Cloud Composer, you can upload a pod YAML to GCS and set it to one of the <a href="https://cloud.google.com/composer/docs/concepts/cloud-storage" rel="nofollow noreferrer">mapped storage paths</a> (e.g. <code>/home/airflow/gcs/dags/my-pod.yaml</code> if you put it in the DAGs directory.</p>
<p>Reading through the <code>airflow.providers.google.cloud</code> version of KubernetesPodOperator, it's possible that <code>full_pod_spec</code> is broken in newer version of the operator. However, it should work with the old contrib version.</p>
