<p>Kubernetes Newbie here.</p>
<p>I was following a <a href="https://www.youtube.com/watch?v=GDOw8ByzMyY" rel="nofollow noreferrer">youtube tutorial from Marc Lamberti</a> on how to install airflow using Kubernetes locally - using Kind. I was able to create the k8s cluster using below commands:</p>
<pre><code>kind create cluster --name airflow-cluster --config kind-cluster.yaml

helm repo add apache-airflow https://airflow.apache.org
helm repo update

kubectl create namespace airflow
helm install airflow apache-airflow/airflow --namespace airflow --debug

# Port forward 
kubectl port-forward svc/airflow-webserver 8080:8080 -n airflow --context kind-airflow-cluster
</code></pre>
<p>The next part was to upgrade the generate values.yaml using below command</p>
<pre><code>helm show values apache-airflow/airflow &gt; values.yaml
</code></pre>
<p>The next step was to modify this values.yaml and run a <code>helm upgrade</code>, however, whether I modify the values.yaml file or not, I am getting the below error:</p>
<pre><code>helm upgrade --install airflow apache-airflow/airflow -n airflow -f values.yaml --debug
history.go:56: [debug] getting history for release airflow
upgrade.go:144: [debug] preparing upgrade for airflow
Error: UPGRADE FAILED: execution error at (airflow/charts/postgresql/templates/secrets.yaml:20:15):
PASSWORDS ERROR: The secret &quot;airflow-postgresql&quot; does not contain the key &quot;password&quot;

helm.go:84: [debug] execution error at (airflow/charts/postgresql/templates/secrets.yaml:20:15):
PASSWORDS ERROR: The secret &quot;airflow-postgresql&quot; does not contain the key &quot;password&quot;

UPGRADE FAILED
main.newUpgradeCmd.func2
    helm.sh/helm/v3/cmd/helm/upgrade.go:203
github.com/spf13/cobra.(*Command).execute
    github.com/spf13/cobra@v1.6.1/command.go:916
github.com/spf13/cobra.(*Command).ExecuteC
    github.com/spf13/cobra@v1.6.1/command.go:1044
github.com/spf13/cobra.(*Command).Execute
    github.com/spf13/cobra@v1.6.1/command.go:968
main.main
    helm.sh/helm/v3/cmd/helm/helm.go:83
runtime.main
    runtime/proc.go:250
runtime.goexit
    runtime/asm_arm64.s:1172
</code></pre>
<p>On examining the secret <code>airflow-postgresql</code>, I found that it had <code>postgres-password</code> in data, but not <code>password</code>, so I added it.</p>
<pre><code>$ k describe secret airflow-postgresql
Name:         airflow-postgresql
Namespace:    airflow
Labels:       app.kubernetes.io/instance=airflow
              app.kubernetes.io/managed-by=Helm
              app.kubernetes.io/name=postgresql
              helm.sh/chart=postgresql-12.1.9
Annotations:  meta.helm.sh/release-name: airflow
              meta.helm.sh/release-namespace: airflow

Type:  Opaque

Data
====
postgres-password:  8 bytes
</code></pre>
<p>How I added the password -</p>
<pre><code>export POSTGRES_PASS=$(kubectl get secret airflow-postgresql -o jsonpath=&quot;{.data.postgres-password}&quot; | base64 --decode)

k create secret generic airflow-postgresql --from-literal=password=$POSTGRES_PASS --dry-run=client -o yaml | k apply -f -
</code></pre>
<p>I added the password data to the secret and re-tried. This time I did not have the same error, but the postgres pod went into <code>CrashLoop Backoff</code> and I can see in the logs of the pod that the pod is shutting down.</p>
<pre><code>│ postgresql 23:36:12.33 INFO  ==&gt; ** Starting PostgreSQL **                                                          ││ 2023-04-04 23:36:12.348 GMT [1] LOG:  pgaudit extension initialized                                                 │
│ 2023-04-04 23:36:12.349 GMT [1] LOG:  listening on IPv4 address &quot;0.0.0.0&quot;, port 5432                                ││ 2023-04-04 23:36:12.349 GMT [1] LOG:  listening on IPv6 address &quot;::&quot;, port 5432                                     │
│ 2023-04-04 23:36:12.350 GMT [1] LOG:  listening on Unix socket &quot;/tmp/.s.PGSQL.5432&quot;                                 ││ 2023-04-04 23:36:12.356 GMT [95] LOG:  database system was shut down at 2023-04-04 23:36:12 GMT                     │
│ 2023-04-04 23:36:12.360 GMT [1] LOG:  database system is ready to accept connections                                ││ 2023-04-04 23:37:42.055 GMT [1] LOG:  received smart shutdown request                                               │
│ 2023-04-04 23:37:42.066 GMT [1] LOG:  background worker &quot;logical replication launcher&quot; (PID 101) exited with exit c ││ 2023-04-04 23:37:42.066 GMT [96] LOG:  shutting down                                                                │
│ 2023-04-04 23:37:42.076 GMT [1] LOG:  database system is shut down                                                  ││ Stream closed EOF for airflow/airflow-postgresql-0 (postgresql)                                                     │
</code></pre>
<p>Due to the above, other pods - scheduler, webserver are not able to connect to the DB and are in <code>CrashLoopBackoff</code> state as well. (I could see in their logs that there were last trying to connect to the DB)</p>
<p>I am not sure what I need to do to be able to use values.yaml to upgrade the helm chart to a new version.</p>
<p>Many thanks for reading so far. Any guidance would be really appreciated
Thank you!</p>

Moni
<p>According to the official manual <a href="https://helm.sh/docs/helm/helm_show_values/" rel="nofollow noreferrer">https://helm.sh/docs/helm/helm_show_values/</a>, <code>helm show values</code> only show the original values based on its original <code>chart</code>.</p>
<p>Since you are not specifying the chart version, your <code>install</code> command might use different version from <code>install --upgrade</code> command.</p>
<p>The right way to upgrade your installation is get <code>helm release</code> values from your installation and <code>compare</code> it with the <code>values.yaml</code> from new <code>chart version</code> and adjust the new required <code>value</code>.</p>
<p>Some step might be helpful:</p>
<h5>Step1: Get the chart version you've installed: <code>helm -n airflow list</code>, you'll see something as following(my release <code>tooljet</code> for example):</h5>
<pre><code>tooljet         it              12              2023-03-17 21:03:42.9213007 +0800 CST   deployed        app-0.7.7               0.1.0
</code></pre>
<blockquote>
<p>Here <code>app-0.7.7</code> means chart name is <code>app</code> and chart version is <code>0.7.7</code>.</p>
</blockquote>
<h5>Step2: Fetch the chart values of your previous installed helm release: <code>helm show values apache-airflow/airflow --version=0.7.7 &gt; values-old.yaml</code> (remember to change <code>0.7.7</code> to your <code>helm release</code> version as showed in <code>step1</code>)</h5>
<h5>Step3: Fetch the latest chart vaues: <code>helm show values apache-airflow/airflow &gt; values-new.yaml</code></h5>
<h5>Step4: Compared <code>values-old.yaml</code> and <code>values-new.yaml</code> to find out what <code>values</code> are newly required from new chart. Now you can add missing values in <code>values-old.yaml</code> and using it in your <code>helm upgrade --install airflow apache/airflow -f values-old.yaml</code> command.</h5>
<hr />
<p><strong>WARN</strong>: If you've changed the chart values, you'd better to merge <code>values.yaml</code> from your <code>helm release</code>, you can get its effective <code>values</code> from command: <code>helm -n airflow get values airflow &gt; values-effect.yaml</code>. Now your final values should contains values merged from <code>values-old.yaml</code>, <code>values-effect.yaml</code> and <code>missing new required values</code>.</p>
