<p>I am increasing the persistent volume of our <strong>INT</strong> database from <strong>30Gi</strong> to <strong>40Gi</strong> upon releasing helm changes I am getting below error:</p>
<blockquote>
<p>Error: UPGRADE FAILED: cannot patch &quot;dbname-dbname-db&quot; with kind StatefulSet: StatefulSet.apps &quot;dbname-dbname-db&quot; is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', and 'updateStrategy' are forbidden</p>
</blockquote>
<p>I did a bit of research and I am figuring out that I need to stop the pod connecting to the database and/or delete the current service name <code>dbname-dbname-db</code> to apply the changes.</p>
<p>This seems to be pausing risks of <em><strong>downtime</strong></em> in our environment.</p>
<p>Can you please suggest possible ways of updating PVC capacity without downtime in the environment?</p>
<p>I read about <code>database mirroring</code> but it's not clear to me how to implement this. we run our application in a Kubernetes cluster I would appreciate it if one could show me the <code>kubectl</code> commands to create <code>database mirror</code></p>
<p>below is part of the values.YAML.gotmpl file for the helm.</p>
<pre><code>volumeMounts: {{- toYaml .volumeMounts | nindent 4 }}
  volumes: {{- toYaml .volumes | nindent 4 }}

  postgresql:
    postgresqlUsername: &quot;user1&quot;
    postgresqlDatabase: {{ .database.name | quote }}
    existingSecret: {{ .database.existingSecret | quote }}
    persistence:
      enabled: true
      size: 30Gi  ----&gt; value being changed to **40Gi**
    nameOverride: &quot;dbname-db&quot;
    metrics:
      enabled: true
    networkPolicy:
      enabled: false
      allowExternal: true

  image:
    replicaCount: {{ .image.replicaCount }}
    repository: itops.company.io
    imageName: company/app-backend
    tag: {{ .image.tag | quote }}
</code></pre>

Siya Nxuseka
<p>Let's divide your issue into two different issues:</p>
<ol>
<li>Increase your volume size</li>
<li>Update the value in your <code>values.YAML.gotmpl</code> file to reflect the actual size.</li>
</ol>
<p>According to the error you get, it is apparently that the size in the values file is templated to the statefulset configuration under the field <code>volumeClaimTemplates</code>. However, currently this field cannot be patched (currently (April 2022) there is an <a href="https://github.com/kubernetes/enhancements/issues/661" rel="nofollow noreferrer">open issue in kubernetes about it</a>).</p>
<p>The way to resize the PVC is by patching the PVC itself.</p>
<p>Kubernetes can let you increase the PVC size without the need to restart the pod that uses it. This depends on what version of kubernetes you have, what type of storage and what flags are set by the cluster administrator.</p>
<p>You can try the following, supposing you have a kubectl access to the namespace of your database:</p>
<ol>
<li>Figure out the number of pods in your statefulset: <code>kubectl get sts dbname-dbname-db</code></li>
<li>Figure out the PVC names by runing <code>kubectl describe statefulset dbname-dbname-db</code>. You should see a section <code>Volume Claims</code>. the <code>name</code> field in this section is the prefix for the PVC name. the actual PVC name should be <code>&lt;prefix&gt;-dbname-dbname-db-&lt;i&gt;</code> when <code>i</code> is the serial number of the pod (out of the number of the pods for this statefulset), starting with 0.</li>
<li>For each PVC, run <code>kubectl patch pvc &lt;pvc-name&gt; -p '{&quot;spec&quot;:{&quot;resources&quot;:{&quot;requests&quot;:{&quot;storage&quot;:&quot;40Gi&quot;}}}}}'</code></li>
</ol>
<p>If the previous commands are successful, you can track the status of the resizing process by running <code>kubectl describe pvc &lt;pvc-name&gt;</code> and looking in the <code>conditions</code> field. It can take several minutes for the process to complete, make sure it is completed before proceeding with the next steps.</p>
<p>The above process leaves you with the second problem - the code in your values file is not aligned with the actual volume size.</p>
<p>The solution for this problem is as follows, and must be done <strong>after</strong> the previous process is completed :</p>
<ol>
<li>Make sure you are ready to deploy your changes to the values file (30Gi -&gt; 40Gi).</li>
<li>Delete the statefulset object, without deleting any running pod, by running <code>kubectl delete statefulset dbname-dbname-db --cascade=orphan</code>. this way, the working pods continue to work without interruption.</li>
<li>Deploy your changes. The statefulset will be created again and should be automatically associated with the running pod(s).</li>
</ol>
<p>Reference:
<a href="https://kubernetes.io/docs/concepts/storage/persistent-volumes/#expanding-persistent-volumes-claims" rel="nofollow noreferrer">Expanding persistent volumes claims</a></p>
