<p>I am using the ECK <a href="https://github.com/elastic/cloud-on-k8s" rel="nofollow noreferrer">operator</a>, to create an <code>Elasticsearch</code> instance.</p>
<p>The instance uses a <code>StorageClass</code> that has <code>Retain</code> (instead of <code>Delete</code>) as its reclaim policy.</p>
<p>Here are my <code>PVC</code>s <strong>before</strong> deleting the <code>Elasticsearch</code> instance</p>
<pre><code>▶ k get pvc
NAME                                                   STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS              AGE
elasticsearch-data--es-multirolenodes1-0   Bound    pvc-ba157213-67cf-4b81-8fe2-6211b771e62c   20Gi       RWO               balanced-retain-csi   8m15s
elasticsearch-data--es-multirolenodes1-1   Bound    pvc-e77dbb00-7cad-419f-953e-f3398e3860f4   20Gi       RWO               balanced-retain-csi   7m11s
elasticsearch-data--es-multirolenodes1-2   Bound    pvc-b258821b-0d93-4ea3-8bf1-db590b93adfd   20Gi       RWO               balanced-retain-csi   6m5s
</code></pre>
<p>I deleted and re-created the <code>Elasticsearch</code> instance with the hope that due to the <code>Retain</code> policy, the new pods (i.e. their <code>PVC</code>s would bind to the existing <code>PV</code>s (and data wouldn't get lost)</p>
<p>However now my pods of the <code>nodeSet</code> are all in pending state with this error</p>
<pre><code>Events:
  Type     Reason             Age                  From                Message
  ----     ------             ----                 ----                -------
  Warning  FailedScheduling   2m37s                default-scheduler   persistentvolumeclaim &quot;elasticsearch-data--es-multirolenodes1-0&quot; is being deleted
  Normal   NotTriggerScaleUp  2m32s                cluster-autoscaler  pod didn't trigger scale-up: 2 persistentvolumeclaim &quot;elasticsearch-data--es-multirolenodes1-0&quot; not found
  Warning  FailedScheduling   12s (x7 over 2m37s)  default-scheduler   persistentvolumeclaim &quot;elasticsearch-data--es-multirolenodes1-0&quot; not found
</code></pre>
<p>Why is this happening?</p>
<p><strong>edit</strong>: Here are the corresponding <code>PV</code>s</p>
<pre><code>▶ k get pv 
pvc-b258821b-0d93-4ea3-8bf1-db590b93adfd   20Gi       RWO            Retain           Released   elastic/elasticsearch-data--es-multirolenodes1-2          balanced-retain-csi            20m
pvc-ba157213-67cf-4b81-8fe2-6211b771e62c   20Gi       RWO            Retain           Released   elastic/elasticsearch-data--es-multirolenodes1-0          balanced-retain-csi            22m
pvc-e77dbb00-7cad-419f-953e-f3398e3860f4   20Gi       RWO            Retain           Released   elastic/elasticsearch-data--es-multirolenodes1-1          balanced-retain-csi            21m
</code></pre>
<p>There is of course no <code>PVC</code> now</p>
<pre><code>▶ k get pvc                       
No resources found in elastic namespace.
</code></pre>
<p>The <code>StorageClass</code> under consideration is using the <code>csi</code> driver for the GCP persistent disk, fwiw</p>

pkaramol
<blockquote>
<p>with the hope that due to the Retain policy, the new pods (i.e. their PVCs would bind to the existing PVs (and data wouldn't get lost)</p>
</blockquote>
<p>It is explicitly written in the <a href="https://kubernetes.io/docs/concepts/storage/persistent-volumes/#retain" rel="nofollow noreferrer">documentation</a> that this is not what happens. the PVs are <strong>not</strong> available for another PVC after delete of a PVC.</p>
<blockquote>
<p>the PersistentVolume still exists and the volume is considered &quot;released&quot;. But it is not yet available for another claim because the previous claimant's data remains on the volume.</p>
</blockquote>
