<p>We are building workflow scheduling application. We found Airflow as a good option for workflow manager and Kubernetes as good option for Cluster manager. Thus, flow would be, </p>

<ol>
<li>We will submit workflow DAG to Airflow.</li>
<li>Airflow should submit the tasks of a given DAG to Kubernetes by specifying docker image.</li>
<li>Kubernetes should execute the task by running docker container on an available EC2 worker node of a cluster. </li>
</ol>

<p>On searching, we found, Airflow has Operators for integrating with ECS, Mesos but not for Kubernetes. However, we found a request for Kubernetes Operator on <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=71013666" rel="noreferrer">Airflow wiki</a>, but not any further update on it. </p>

<p>So, the question to be simply put is, how to integrate Airflow with Kubernetes?</p>

<p>This is in flight right now. You just can follow along with <a href="https://issues.apache.org/jira/browse/AIRFLOW-1314" rel="noreferrer">this major jira ticket</a></p>

<p>One of the more stable branches (work is being led by a lot of this team) is located in the <a href="https://github.com/bloomberg/airflow/tree/airflow-kubernetes-executor" rel="noreferrer">bloomberg fork on github in the airflow-kubernetes-executor branch</a> though it is in the process of being rebased off of a constantly moving airflow master.</p>

<p>I have a branch on my fork that addresses many of the short term issues and runs well enough called <a href="https://github.com/jzucker2/airflow/tree/frankensteins-monster" rel="noreferrer">frankensteins-monster</a>. Use this at your own risk though it works for me right now. I am building a docker image using the <code>build.sh</code> script located in <code>scripts/ci/kubernetes/docker</code>.</p>

<p>Good luck!</p>
