<p>My objective is to be able to deploy airflow on Kubernetes using a custom image (placed in ECR. The reason I want to use this custom Image is because I want to deploy another tool (<a href="https://github.com/airflow-helm/charts/blob/main/charts/airflow/values.yaml#L9" rel="nofollow noreferrer">dbt</a>) with airflow in the same container (also open for other suggestions there)</p>
<p><strong>What actually worked:</strong>
I have managed to use <a href="https://github.com/airflow-helm/charts" rel="nofollow noreferrer">this Helm chart</a> (which uses the following <a href="https://github.com/airflow-helm/charts/blob/main/charts/airflow/values.yaml#L9" rel="nofollow noreferrer">image</a> as default) to deploy</p>
<p><strong>What I tried to do and did not work:</strong>
I wanted to now exchange the default <a href="https://github.com/airflow-helm/charts/blob/main/charts/airflow/values.yaml#L9" rel="nofollow noreferrer">image</a> with my custom image in ECR, so I created <code>values.yaml</code>, that contains:</p>
<pre><code>airflow:
  image:
    repository: 000000000.dkr.ecr.eu-central-1.amazonaws.com/foo/meow
    tag: latest
</code></pre>
<p>And then ran:</p>
<pre><code>helm upgrade airflow-pod airflow-stable/airflow --version &quot;7.14.0&quot; --values values.yaml
</code></pre>
<p>Which I expected to override the default yaml and pull the image from ECR instead. I then ran <code>describe pod airflow-pod</code>, and found the following log of the error (snippet):</p>
<pre><code>  Type     Reason     Age                   From               Message
  ----     ------     ----                  ----               -------
  Normal   Scheduled  12m                   default-scheduler  Successfully assigned 000000000.dkr.ecr.eu-central-1.amazonaws.com/foo/meow to ip-10-0-0-0eu-central-1.compute.internal
  Normal   Pulling    12m                   kubelet            Pulling image &quot;000000000.dkr.ecr.eu-central-1.amazonaws.com/foo/meow:latest&quot;
  Normal   Pulled     11m                   kubelet            Successfully pulled image &quot;000000000.dkr.ecr.eu-central-1.amazonaws.com/foo/meow:latest&quot;
  Normal   Created    9m39s (x5 over 11m)   kubelet            Created container airflow-web
  Normal   Pulled     9m39s (x4 over 11m)   kubelet            Container image &quot;000000000.dkr.ecr.eu-central-1.amazonaws.com/foo/meow:latest&quot; already present on machine
  Warning  Failed     9m38s (x5 over 11m)   kubelet            Error: failed to start container &quot;airflow-web&quot;: Error response from daemon: OCI runtime create failed: container_linux.go:370: starting container process caused: exec: &quot;/usr/bin/dumb-init&quot;: stat /usr/bin/dumb-init: no such file or directory: unknown
</code></pre>
<p><strong>What I have tried to confirm/fix the issue</strong></p>
<p>First, I tried to see if it is an ECR issue. So I put the same <a href="https://github.com/airflow-helm/charts/blob/main/charts/airflow/values.yaml#L9" rel="nofollow noreferrer">original image</a> in ECR (instead of my own image with dbt), and found that the same error above persists.</p>
<p>Second, I digged and found the following <a href="https://stackoverflow.com/questions/61955393/weird-error-in-kubernetes-starting-container-process-caused-exec-usr-bin">question</a> that makes me think I can't use that airflow helm <a href="https://github.com/airflow-helm/charts/tree/main/charts/airflow" rel="nofollow noreferrer">chart</a> from an ECR repo (non-official one).</p>
<p><strong>One last approach I took as an alternative path:</strong>
I tried to use the <a href="https://github.com/apache/airflow/tree/master/chart" rel="nofollow noreferrer">chart on the Apache airflow repo</a> like:</p>
<pre><code>helm install airflow . --namespace airflow-deploy --set executor=CeleryExecutor --set workers.keda.enabled=true --set workers.persistence.enabled=false
</code></pre>
<p>But I got the error:</p>
<pre><code>Error: failed post-install: timed out waiting for the condition
</code></pre>

<p>The original image seems to have &quot;dumb-init&quot; binary in, so it should work. However, if you use &quot;imagePullPolicy: IfNotPresent&quot; then Kubernetes might cache the image and even if you re-upload a new image to ECR it might not be pulled (though I believe for latest it should be, unless some custom configuration of the Kubernetes is in place.</p>
<p>See <a href="https://kubernetes.io/docs/concepts/containers/images/#updating-images" rel="nofollow noreferrer">https://kubernetes.io/docs/concepts/containers/images/#updating-images</a></p>
<p>You can always run the image and check it's content locally. The official Docker image of Airflow has support for <code>bash</code> command:</p>
<pre><code>docker run -it apache/airflow:1.10.12-python3.6 bash
airflow@18278a339579:/opt/airflow$ /usr/bin/dumb-init --help
dumb-init v1.2.2
Usage: /usr/bin/dumb-init [option] command [[arg] ...]
dumb-init is a simple process supervisor that forwards signals to children.
It is designed to run as PID1 in minimal container environments.
Optional arguments:
   -c, --single-child   Run in single-child mode.
                        In this mode, signals are only proxied to the
                        direct child and not any of its descendants.
   -r, --rewrite s:r    Rewrite received signal s to new signal r before proxying.
                        To ignore (not proxy) a signal, rewrite it to 0.
                        This option can be specified multiple times.
   -v, --verbose        Print debugging information to stderr.
   -h, --help           Print this help message and exit.
   -V, --version        Print the current version and exit.
Full help is available online at https://github.com/Yelp/dumb-init
</code></pre>
