<p>I am trying to get Airflow Celery (running on docker-compose) to start a K8s pod with the k8s task operator like:</p>
<pre><code>@task.kubernetes(
    image=&quot;python:3.11.5-alpine3.18&quot;,
    name=&quot;airflow_k8s_task&quot;,
    namespace=&quot;default&quot;,
    config_file=&quot;/opt/airflow/dags/k8s_config.yaml&quot;,
    cluster_context=&quot;kind-kind&quot;,
)
def k8s_task():
    print(f&quot;runing in k8s! Python version: {sys.version_info}&quot;)
</code></pre>
<p>For the k8s cluster, I just used <code>kind</code> to setup a local cluster. Then, copied the config from <code>~/.kube/config</code> into the airflow container via volume mount. But I get error like:</p>
<pre><code>[2023-09-30, 13:28:01 UTC] {pod.py:961} INFO - Building pod ***-k8s-task-lugfpzol with labels: {'dag_id': 'custom_docker_k8s_dag', 'task_id': 'k8s_task', 'run_id': 'manual__2023-09-30T111956.4977000000-0da65ab9a', 'kubernetes_pod_operator': 'True', 'try_number': '7'}
[2023-09-30, 13:28:01 UTC] {connectionpool.py:823} WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('&lt;urllib3.connection.HTTPSConnection object at 0x7f57b662b550&gt;: Failed to establish a new connection: [Errno 111] Connection refused')': /api/v1/namespaces/default/pods?labelSelector=dag_id%3Dcustom_docker_k8s_dag%2Ckubernetes_pod_operator%3DTrue%2Crun_id%3Dmanual__2023-09-30T111956.4977000000-0da65ab9a%2Ctask_id%3Dk8s_task%2Calready_checked%21%3DTrue%2C%21***-worker
[2023-09-30, 13:28:01 UTC] {connectionpool.py:823} WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('&lt;urllib3.connection.HTTPSConnection object at 0x7f57b662b3d0&gt;: Failed to establish a new connection: [Errno 111] Connection refused')': /api/v1/namespaces/default/pods?labelSelector=dag_id%3Dcustom_docker_k8s_dag%2Ckubernetes_pod_operator%3DTrue%2Crun_id%3Dmanual__2023-09-30T111956.4977000000-0da65ab9a%2Ctask_id%3Dk8s_task%2Calready_checked%21%3DTrue%2C%21***-worker
[2023-09-30, 13:28:01 UTC] {connectionpool.py:823} WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('&lt;urllib3.connection.HTTPSConnection object at 0x7f57b662b090&gt;: Failed to establish a new connection: [Errno 111] Connection refused')': /api/v1/namespaces/default/pods?labelSelector=dag_id%3Dcustom_docker_k8s_dag%2Ckubernetes_pod_operator%3DTrue%2Crun_id%3Dmanual__2023-09-30T111956.4977000000-0da65ab9a%2Ctask_id%3Dk8s_task%2Calready_checked%21%3DTrue%2C%21***-worker
[2023-09-30, 13:28:01 UTC] {pod.py:1017} ERROR - 'NoneType' object has no attribute 'metadata'
</code></pre>
<p>I also tried setting</p>
<pre><code>cluster_context=&quot;kind-kind&quot;,
</code></pre>
<p>Also for my airflow worker, I ran with <code>network_mode: host</code> so that it can connect to my local host machine</p>

Jiew Meng
<p>You cannot simply pass a <code>.kubeconfig</code> file in this manner without modification.</p>
<p>Kind will create (or edit) a <code>.kubeconfig</code> file for you which refers to the <em>host</em> IP address of the cluster. Regardless of how you set your <code>network_mode</code>, you will not be able to access the <code>Host</code> network as you would from your local machine. There are numerous reasons for this, and the KinD team have multiple GitHub issues that deal with the complexities of trying to pass around networking information through the config file. The net effect of most of these is <strong>don't do it</strong>.</p>
<p>You have a few options to manage this, the simplest is probably to use the <a href="https://airflow.apache.org/docs/helm-chart/stable/index.html" rel="nofollow noreferrer">Airflow-supplied Helm Chart</a> which will configure Airflow to run inside your cluster rather than passing your <code>.kubeconfig</code> into some random pod running in a different Docker network.</p>
<p>This chart automatically handles the creation of the necessary service accounts and kubernetes Role-Based Access Control for your pod to run correctly.</p>
<p>EDIT: The intent of the question is more or less to use Airflow running in Docker-Compose to launch something within your KinD cluster.</p>
<p>If this is truly your intention, you need to do three things:</p>
<ul>
<li><p>Set your container to run in the <code>kind</code> network in your Docker-Compose file by <a href="https://docs.docker.com/compose/networking/#specify-custom-networks" rel="nofollow noreferrer">specifying a custom network</a> (KinD creates this network when you launch a cluster)</p>
</li>
<li><p>Update your <code>.kubeconfig</code> file to point to the <strong>kind network address</strong> of the cluster rather than localhost</p>
</li>
<li><p>Pass this config to your container, which is now running in the same network as the KinD control plane, and now has the KinD network address for it</p>
</li>
</ul>
<p>The second point is achieved by using <code>docker inspect</code> on the KinD control plane node and obtaining the private IP address for the control plane container. Docker documentation <a href="https://docs.docker.com/engine/reference/commandline/inspect/#get-an-instances-ip-address" rel="nofollow noreferrer">describes this here</a>. Once you have the IP address of the pod in the <code>kind</code> network, you can replace the value of the IP address in the <code>Server</code> field of the configuration for your target cluster.</p>
<p>You can also alter the properties of the KinD control plane by <a href="https://kind.sigs.k8s.io/docs/user/configuration/#getting-started" rel="nofollow noreferrer">passing a configuration file to KinD</a> and setting the <a href="https://kind.sigs.k8s.io/docs/user/configuration/#api-server" rel="nofollow noreferrer">API Server address</a> but this is not recommended by the KinD team.</p>
<p>A final comment -
You are better to use <code>kind get kubeconfig</code> and then patch the <code>Server</code> field in the returned value. Passing your local <code>.kubeconfig</code> may have all sorts of issues such as multiple contexts or configurations that will make it a nightmare to work with long-term.</p>
