<p>I am still learning about Airflow. My objective is to deploy Airflow on Kubernetes (I am using AWS EKS) with <code>CeleryExecutor</code> and using Helm charts. As a start, I am primarily focused on deploying the example dags.</p>
<p>I have installed the chart like:</p>
<pre class="lang-sh prettyprint-override"><code>helm install airflow bitnami/airflow \
    --set airflow.loadExamples=true \
    --set web.baseUrl=http://127.0.0.1:8080 \
    --set auth.username=$AIRFLOW_USER  \
    --set auth.password=$AIRFLOW_PASSWORD \
    --set auth.fernetKey=$AIRFLOW_FERNETKEY = \
    --set postgresql.postgresqlPassword=$POSTGRESQL_PASSWORD \
    --set redis.password=$REDIS_PASSWORD 
</code></pre>
<p>I have been successful in deploying airflow (using the <a href="https://github.com/bitnami/charts/tree/master/bitnami/airflow" rel="nofollow noreferrer">bitnami chart</a> but also had the same issue with other charts), and by running <code>kubectl get pods</code> I get:</p>
<pre class="lang-sh prettyprint-override"><code>NAME                                                 READY   STATUS    RESTARTS   AGE
bitnami-release-airflow-scheduler-774d647447-j6vpd   1/1     Running   0          4m6s
bitnami-release-airflow-web-5897c99754-hq6nr         1/1     Running   0          4m6s
bitnami-release-airflow-worker-0                     0/1     Running   0          4m6s
bitnami-release-postgresql-0                         1/1     Running   0          4m6s
bitnami-release-redis-master-0                       1/1     Running   0          4m6s
</code></pre>
<p>I have port-forwarded from the web server's pod to my machine, however, when I try to run the example DAGs, the tasks seem to be stuck in <code>running</code>.  I have later found an error on the UI: <code>The scheduler does not appear to be running. The last heartbeat was received 1 minute ago.</code> which seems to me that the scheduler is down.</p>
<p>Things I have tried:</p>
<ol>
<li>Manually running <code>init db</code> on the webserver via: <code>kubectl exec -it  bitnami-release-airflow-web-5897c99754-hq6nr -- bash -c &quot;airflow initdb&quot;</code> (was successful but did not solve the problem)</li>
<li>Manually running <code>airflow scheduler</code>  via: <code>kubectl exec -it  bitnami-release-airflow-scheduler-774d647447-j6vpd -- bash -c &quot;airflow scheduler</code> (Tasks started getting added to the queue but then got: <code>ERROR - Process timed out, PID: 5957</code></li>
<li>Trying a different Helm chart altogether (tried the <a href="https://github.com/airflow-helm/charts/tree/main/charts/airflow" rel="nofollow noreferrer">airflow-helm chart</a> but had the same issue with tasks stuck in <code>running</code> state)</li>
</ol>
<p>What are the steps that I need to take after installing the Helm chart? Am I required to run commands to start services manually (e.g. the scheduler)?</p>
<p>Happy to provide further information if needed.</p>
<p>Update:
Adding a snippet from the scheduler's logs:</p>
<pre><code>[2020-12-19 11:23:10,487] {scheduler_job.py:1195} INFO - Sending ('example_bash_operator', 'runme_2', datetime.datetime(2020, 12, 19, 11, 15, 41, 826557, tzinfo=&lt;TimezoneInfo [UTC, GMT, +00:00:00, STD]&gt;), 1) to executor with priority 3 and queue default
[2020-12-19 11:23:10,487] {base_executor.py:58} INFO - Adding to queue: ['airflow', 'run', 'example_bash_operator', 'runme_2', '2020-12-19T11:15:41.826557+00:00', '--local', '--pool', 'default_pool', '-sd', '/opt/bitnami/airflow/venv/lib/python3.6/site-packages/airflow/example_dags/example_bash_operator.py']
[2020-12-19 11:23:12,500] {timeout.py:42} ERROR - Process timed out, PID: 5957
</code></pre>

<p>I have managed to solve the issue. One of the nodes was running out of memory, and I thought it was an issue with airflow configurations rather than an issue with Kubernetes.</p>
