<p>I am trying to create pv and pvc resources via <code>kuberctl create -f pv-definition.json</code> but k8s doesn't make any progress and it just kept reporting <code>Pending</code> states for them. </p>

<p>Looked at <code>kubectl get events</code> and logs from <code>api</code> and <code>controller</code>, but I don't see anything related to messages to pv or pvc creation. I have the disk created beforehand on gcloud and I have verified that the name and disk size matches.</p>

<p>The pv definition is very basic: </p>

<pre><code>{
  "apiVersion": "v1",
  "kind": "PersistentVolume",
  "metadata": {
    "name": "test-0b-pv"
  },
  "spec": {
    "accessModes": [
      "ReadWriteOnce"
    ],
    "capacity": {
      "storage": "50Gi"
    },
    "gcePersistentDisk": {
      "fsType": "ext4",
      "pdName": "test-0b"
    },
    "persistentVolumeReclaimPolicy": "Retain"
  }
}
</code></pre>

<p>Does anyone have any suggestion for debugging? I have restarted k8s-master services to no avail. </p>

<p>Most of the above answers wrote a solution from perspective of <code>inconsistent or incorrect</code> storage class especially in case it was not provided in <code>PersistentVolumeClaims</code> however we may have different reason here (described below)</p>
<p><strong>Problem Statement:</strong></p>
<p>If you have deleted <code>PersistentVolumeClaim</code> and then re-create it again with the same definition, it will be <code>Pending</code> forever, why?</p>
<p><strong>Explanation:</strong></p>
<ul>
<li>we have <code>&quot;persistentVolumeReclaimPolicy&quot;: &quot;Retain&quot;</code> in <code>PersistentVolume</code> (the default is retain policy)</li>
<li>In case we have deleted <code>PersistentVolumeClaim</code>, the <code>PersistentVolume</code> still exists and the volume is considered <code>released</code>. But it is not yet available for another <code>claim</code> because the previous claimant's data remains on the volume.</li>
<li>so you need to manually reclaim the volume with the following steps:</li>
</ul>
<p><strong>Solution:</strong></p>
<ol>
<li>Delete the <code>PersistentVolume</code> (associated underlying storage asset/resource like EBS, GCE PD, Azure Disk, ...etc will NOT be deleted, still exists)</li>
<li>(<strong>Optional</strong>) Manually clean up the data on the associated storage asset accordingly</li>
<li>(<strong>Optional</strong>) Manually delete the associated storage asset (EBS, GCE PD, Azure Disk, ...etc)</li>
</ol>
<p>Please <strong>note</strong> that if you still need same data, simply create a new <code>PersistentVolume</code> with same storage asset definition then you should be good to create <code>PersistentVolumeClaim</code> again.</p>
<p>One last thing to mention, <code>Retain</code> is not the only option for <code>persistentVolumeReclaimPolicy</code>, here are some other options that you may need to use or try based on use-case scenarios:</p>
<p><strong>Recycle</strong>: performs a basic scrub on the volume (e.g., rm -rf //*) - makes it available again for a new claim. Only <code>NFS</code> and <code>HostPath</code> support recycling.</p>
<p><strong>Delete</strong>: Associated storage asset such as <code>AWS EBS, GCE PD, Azure Disk, or OpenStack Cinder...etc</code> volume is deleted</p>
<p>For more information, please check <a href="https://kubernetes.io/docs/concepts/storage/persistent-volumes/#:%7E:text=Retain,data%20remains%20on%20the%20volume." rel="noreferrer">kubernetes documentation</a></p>
