<p>I am running a Celery Executor and I'm trying to run some python script in the KubernetesPodOperator. Below are examples of what I have tried that didn't work. What am I doing wrong?</p>
<p>Running sctipt</p>
<pre><code>org_node = KubernetesPodOperator(
    namespace='default',
    image=&quot;python&quot;,
    cmds=[&quot;python&quot;, &quot;somescript.py&quot; &quot;-c&quot;],
    arguments=[&quot;print('HELLO')&quot;],
    labels={&quot;foo&quot;: &quot;bar&quot;},
    image_pull_policy=&quot;Always&quot;,
    name=task,
    task_id=task,
    is_delete_operator_pod=False,
    get_logs=True,
    dag=dag
)
</code></pre>
<p>Running function <code>load_users_into_table()</code></p>
<pre><code>def load_users_into_table(postgres_hook, schema, path):
  gdf = read_csv(path)
  gdf.to_sql('users', con=postgres_hook.get_sqlalchemy_engine(), schema=schema)

org_node = KubernetesPodOperator(
    namespace='default',
    image=&quot;python&quot;,
    cmds=[&quot;python&quot;, &quot;somescript.py&quot; &quot;-c&quot;],
    arguments=[load_users_into_table],
    labels={&quot;foo&quot;: &quot;bar&quot;},
    image_pull_policy=&quot;Always&quot;,
    name=task,
    task_id=task,
    is_delete_operator_pod=False,
    get_logs=True,
    dag=dag
)
</code></pre>

Jonathan
<p>The script <code>somescript.py</code> must be in <code>Docker image</code>.</p>
<p><strong>Step-1</strong>: let's create a image <a href="https://docs.docker.com/develop/develop-images/dockerfile_best-practices/" rel="nofollow noreferrer">https://docs.docker.com/develop/develop-images/dockerfile_best-practices/</a>.</p>
<pre><code>FROM python:3.8

# copy requirement.txt from local to container
COPY requirements.txt requirements.txt
# install dependencies into container (geopandas, sqlalchemy)
RUN pip install -r requirements.txt
# copy the python script from local to container
COPY somescript.py somescript.py
ENTRYPOINT [ &quot;python&quot;, &quot;somescript.py&quot;]
</code></pre>
<p><strong>Step-2</strong>: Build and push the image into <code>public Docker repository</code> <a href="https://hub.docker.com" rel="nofollow noreferrer">https://hub.docker.com</a>.</p>
<p><em>NB:</em> <code>kubernetes_pod_operator</code> looks for image from <code>public docker repo</code></p>
<pre><code># build image
docker build -t my-python-img:latest .
# test if your image works perfectly
docker run my-python-img:latest

# push image.
docker tag my-python-img username/my-python-img
docker push username/my-python-img
docker pull username/my-python-img
</code></pre>
<p><strong>step-3:</strong> Lest's create <code>k8s task</code>.</p>
<pre><code>p = KubernetesPodOperator(
        namespace='default',
        image='username/my-python-img:latest',
        labels={'dag-id': dag.dag_id},
        name='airflow-my-image-pod',
        task_id='load-users',
        in_cluster=False,       #False: local, True: cluster               
        cluster_context='microk8s',  
        config_file='/usr/local/airflow/include/.kube/config',
        is_delete_operator_pod=True,
        get_logs=True,
        dag=dag
    )
</code></pre>
<p>If you don't understand where configuration file comes from, look here: <a href="https://www.astronomer.io/docs/cloud/stable/develop/kubepodoperator-local" rel="nofollow noreferrer">https://www.astronomer.io/docs/cloud/stable/develop/kubepodoperator-local</a>.</p>
<p><strong>Finally:</strong> I want to mention something important when working with databases (credentials). Kubernetes offers the use <code>secret</code> to secure sensitive information. <a href="https://airflow.apache.org/docs/apache-airflow-providers-cncf-kubernetes/stable/operators.html" rel="nofollow noreferrer">https://airflow.apache.org/docs/apache-airflow-providers-cncf-kubernetes/stable/operators.html</a></p>
