<p>I have a google cloud composer environment. In my DAG I want to create a pod in GKE. When I come to deploy a simple app based on a docker container that doesn't need any volume configuration or secrets, everything works fine, for example:</p>
<pre><code>kubernetes_max = GKEStartPodOperator(
    # The ID specified for the task.
    task_id=&quot;python-simple-app&quot;,
    # Name of task you want to run, used to generate Pod ID.
    name=&quot;python-demo-app&quot;,
    project_id=PROJECT_ID,
    location=CLUSTER_REGION,
    cluster_name=CLUSTER_NAME,
    # Entrypoint of the container, if not specified the Docker container's
    # entrypoint is used. The cmds parameter is templated.
    cmds=[&quot;python&quot;, &quot;app.py&quot;],
    namespace=&quot;production&quot;,
    image=&quot;gcr.io/path/to/lab-python-job:latest&quot;,
)
</code></pre>
<p>But when I have an application that need to access to my GKE cluster volumes, I need to configure volumes in my pod. The issue is the documentation is not clear regarding this. The only example that I ever <a href="https://airflow.apache.org/docs/apache-airflow-providers-cncf-kubernetes/stable/operators.html" rel="nofollow noreferrer">foud</a> is this:</p>
<pre><code>volume = k8s.V1Volume(
    name='test-volume',
    persistent_volume_claim=k8s.V1PersistentVolumeClaimVolumeSource(claim_name='test-volume'),
)
</code></pre>
<p>While the volumes in the my manifest file (I use it to deploy my app from local) looks like this:</p>
<pre><code>volumes:
  - name: volume-prod
    secret:
      secretName: volume-prod
      items:
        - key: config
          path: config.json
        - key: another_config
          path: another_config.conf
        - key: random-ca
          path: random-ca.pem
</code></pre>
<p>So when I compare how both volumes looks like in the console (when I manually deploy the manifest file that successfully run, and when I deploy the pod using clod composer that fails):</p>
<ul>
<li><p>The successful run - Manifest file:</p>
<p>volume-prod<br>
Name: volume-prod<br>
Type: secret<br>
Source volume identifier: volume-prod</p>
</li>
<li><p>The failed run - Composer <code>GKEStartPodOperator</code>:</p>
<p>volume-prod<br>
Name: volume-prod<br>
Type: emptyDir<br>
Source volume identifier: Node's default medium</p>
</li>
</ul>
<p>How I can configure my pod from cloud composer in a way it can read the volume of my cluster?</p>

Idhem
<p>The <code>KubernetesPodOperator</code>/<code>GKEStartOperator</code> is just a wrapper around the python Kubernetes sdk - I agree that it isn't well documented in the Airflow/Cloud Composer documentation but the Python SDK for Kubernetes itself is well documented.</p>
<p>Start here with the kubernetes python sdk documentation: <a href="https://github.com/kubernetes-client/python/blob/master/kubernetes/docs/V1PodSpec.md" rel="nofollow noreferrer">https://github.com/kubernetes-client/python/blob/master/kubernetes/docs/V1PodSpec.md</a></p>
<p>You'll notice that the arguments the <code>KubernetesPodOperator</code>/<code>GKEStartOperator</code> take match this spec. If you dig into the source code of the operators you'll see that the operator is nothing more than a builder that creates a <code>kubernetes.client.models.V1Pod</code> object and uses the API to deploy the pod.</p>
<p>The <a href="https://airflow.apache.org/docs/apache-airflow-providers-cncf-kubernetes/stable/_api/airflow/providers/cncf/kubernetes/operators/kubernetes_pod/index.html#airflow.providers.cncf.kubernetes.operators.kubernetes_pod.KubernetesPodOperator" rel="nofollow noreferrer">operator</a> takes a <code>volumes</code> parameter which should be of type <code>List[V1Volume]</code>, where the documentation for <code>V1Volume</code> is <a href="https://github.com/kubernetes-client/python/blob/master/kubernetes/docs/V1Volume.md" rel="nofollow noreferrer">here</a>.</p>
<p>So in your case you would need to provide:</p>
<pre class="lang-py prettyprint-override"><code>from kubernetes.client import models as k8s

kubernetes_max = GKEStartPodOperator(
    # The ID specified for the task.
    task_id=&quot;python-simple-app&quot;,
    # Name of task you want to run, used to generate Pod ID.
    name=&quot;python-demo-app&quot;,
    project_id=PROJECT_ID,
    location=CLUSTER_REGION,
    cluster_name=CLUSTER_NAME,
    # Entrypoint of the container, if not specified the Docker container's
    # entrypoint is used. The cmds parameter is templated.
    cmds=[&quot;python&quot;, &quot;app.py&quot;],
    namespace=&quot;production&quot;,
    image=&quot;gcr.io/path/to/lab-python-job:latest&quot;,
    volumes=[
        k8s.V1Volume(
            name=&quot;volume-prod&quot;,
            secret=k8s.V1SecretVolumeSource(
                secret_name=&quot;volume-prod&quot;,
                items=[
                    k8s.V1KeyToPath(key=&quot;config&quot;, path=&quot;config.json&quot;),
                    k8s.V1KeyToPath(key=&quot;another_config&quot;, path=&quot;another_config.conf&quot;),
                    k8s.V1KeyToPath(key=&quot;random-ca&quot;, path=&quot;random-ca.pem&quot;),
                ],
            )
        )
    ]
)
</code></pre>
<p>Alternatively, you can provide your manifest to the <code>pod_template_file</code> argument in <code>GKEStartPodOperator</code> - this will need to be available to the workers inside airflow.</p>
<p>There are 3 ways to create pods in Airflow using this Operator:</p>
<ol>
<li>Use the arguments of the operator to specify what you need and have the operator build the <code>V1Pod</code> for you.</li>
<li>Provide a manifest by passing in <code>pod_template_file</code> argument.</li>
<li>Use the Kubernetes sdk to create a <code>V1Pod</code> object yourself and pass this to the <code>full_pod_spec</code> argument.</li>
</ol>
