<p>I am trying to mount a persistent volume on pods (via a deployment).</p>

<pre><code>apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      containers:
      - image: ...
        volumeMounts:
        - mountPath: /app/folder
          name: volume
      volumes:
      - name: volume
        persistentVolumeClaim:
          claimName: volume-claim
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: volume-claim
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi
</code></pre>

<p>However, the pod stays in "ContainerCreating" status and the events show the following error message.</p>

<pre><code>Unable to mount volumes for pod "podname": timeout expired waiting for volumes to attach or mount for pod "namespace"/"podname". list of unmounted volumes=[volume]. list of unattached volumes=[volume]
</code></pre>

<p>I verified that the persistent volume claim is ok and bound to a persistent volume.</p>

<p>What am I missing here?</p>

znat
<p>When you create a <code>PVC</code> without specifying a <code>PV</code> or type of <code>StorageClass</code> in GKE clusters it will fall back to default option:</p>

<ul>
<li><code>StorageClass: standard</code></li>
<li><code>Provisioner: kubernetes.io/gce-pd</code></li>
<li><code>Type: pd-standard</code></li>
</ul>

<p>Please take a look on official documentation: <a href="https://cloud.google.com/kubernetes-engine/docs/concepts/persistent-volumes" rel="nofollow noreferrer">Cloud.google.com: Kubernetes engine persistent volumes</a></p>

<p>There could be a lot of circumstances that can produce error message encountered. </p>

<p>As it's unknown how many replicas are in your deployment as well as number of nodes and how pods were scheduled on those nodes, I've tried to reproduce your issue and I encountered the same error with following steps (GKE cluster was freshly created to prevent any other dependencies that might affect the behavior).</p>

<p><strong>Steps</strong>: </p>

<ul>
<li>Create a PVC</li>
<li>Create a Deployment with <code>replicas &gt; 1</code></li>
<li>Check the state of pods </li>
<li>Additional links</li>
</ul>

<h3>Create a PVC</h3>

<p>Below is example <code>YAML</code> definition of a <code>PVC</code> the same as yours: </p>

<pre class="lang-yaml prettyprint-override"><code>apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: volume-claim
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi
</code></pre>

<p>After applying above definition please check if it created successfully. You can do it by using below commands: </p>

<ul>
<li><code>$ kubectl get pvc volume-claim</code></li>
<li><code>$ kubectl get pv</code></li>
<li><code>$ kubectl describe pvc volume-claim</code></li>
<li><code>$ kubectl get pvc volume-claim -o yaml</code></li>
</ul>

<h3>Create a Deployment with <code>replicas &gt; 1</code></h3>

<p>Below is example <code>YAML</code> definition of deployment with <code>volumeMounts</code> and <code>replicas</code> > 1: </p>

<pre class="lang-yaml prettyprint-override"><code>apiVersion: apps/v1 
kind: Deployment
metadata:
  name: ubuntu-deployment
spec:
  selector:
    matchLabels:
      app: ubuntu
  replicas: 10 # amount of pods must be &gt; 1
  template:
    metadata:
      labels:
        app: ubuntu
    spec:
      containers:
      - name: ubuntu
        image: ubuntu
        command:
        - sleep
        - "infinity"
        volumeMounts:
        - mountPath: /app/folder
          name: volume
      volumes:
      - name: volume
        persistentVolumeClaim:
          claimName: volume-claim
</code></pre>

<p>Apply it and wait for a while. </p>

<h3>Check the state of pods</h3>

<p>You can check the state of pods with below command: </p>

<p><code>$ kubectl get pods -o wide</code></p>

<p>Output of above command: </p>

<pre class="lang-sh prettyprint-override"><code>NAME                      READY   STATUS              RESTARTS   AGE     IP            NODE                              
ubuntu-deployment-2q64z   0/1     ContainerCreating   0          4m27s   &lt;none&gt;        gke-node-1  
ubuntu-deployment-4tjp2   1/1     Running             0          4m27s   10.56.1.14    gke-node-2   
ubuntu-deployment-5tn8x   0/1     ContainerCreating   0          4m27s   &lt;none&gt;        gke-node-1   
ubuntu-deployment-5tn9m   0/1     ContainerCreating   0          4m27s   &lt;none&gt;        gke-node-3  
ubuntu-deployment-6vkwf   0/1     ContainerCreating   0          4m27s   &lt;none&gt;        gke-node-1  
ubuntu-deployment-9p45q   1/1     Running             0          4m27s   10.56.1.12    gke-node-2  
ubuntu-deployment-lfh7g   0/1     ContainerCreating   0          4m27s   &lt;none&gt;        gke-node-3  
ubuntu-deployment-qxwmq   1/1     Running             0          4m27s   10.56.1.13    gke-node-2 
ubuntu-deployment-r7k2k   0/1     ContainerCreating   0          4m27s   &lt;none&gt;        gke-node-3   
ubuntu-deployment-rnr72   0/1     ContainerCreating   0          4m27s   &lt;none&gt;        gke-node-3
</code></pre>

<p>Take a look on above output:</p>

<ul>
<li>3 pods are in <code>Running</code> state</li>
<li>7 pods are in <code>ContainerCreating</code> state </li>
</ul>

<p><strong>All of the <code>Running</code> pods are located on the same <code>gke-node-2</code></strong></p>

<p>You can get more detailed information why pods are in <code>ContainerCreating</code> state by:</p>

<p><code>$ kubectl describe pod NAME_OF_POD_WITH_CC_STATE</code></p>

<p>The <code>Events</code> part in above command shows: </p>

<pre class="lang-sh prettyprint-override"><code>Events:
  Type     Reason              Age                From                                             Message
  ----     ------              ----               ----                                             -------
  Normal   Scheduled           14m                default-scheduler                                Successfully assigned default/ubuntu-deployment-2q64z to gke-node-1
  Warning  FailedAttachVolume  14m                attachdetach-controller                          Multi-Attach error for volume "pvc-7d756147-6434-11ea-a666-42010a9c0058" Volume is already used by pod(s) ubuntu-deployment-qxwmq, ubuntu-deployment-9p45q, ubuntu-deployment-4tjp2
  Warning  FailedMount         92s (x6 over 12m)  kubelet, gke-node-1  Unable to mount volumes for pod "ubuntu-deployment-2q64z_default(9dc28e95-6434-11ea-a666-42010a9c0058)": timeout expired waiting for volumes to attach or mount for pod "default"/"ubuntu-deployment-2q64z". list of unmounted volumes=[volume]. list of unattached volumes=[volume default-token-dnvnj]
</code></pre>

<p>Pod cannot pass <code>ContainerCreating</code> state because of failed mounting of a <code>volume</code>. Mentioned <code>volume</code> is already used by other pods on a different node. </p>

<blockquote>
  <p><strong>ReadWriteOnce:</strong> The Volume can be mounted as read-write by a single node.</p>
</blockquote>

<h3>Additional links</h3>

<p>Please take a look at: <a href="https://cloud.google.com/kubernetes-engine/docs/concepts/persistent-volumes#access_modes" rel="nofollow noreferrer">Cloud.google.com: Access modes of persistent volumes</a>. </p>

<p>There is detailed answer on topic of access mode: <a href="https://stackoverflow.com/a/60308557">Stackoverflow.com: Why can you set multiple accessmodes on a persistent volume</a></p>

<p>As it's unknown what you are trying to achieve please take a look on comparison between Deployments and Statefulsets: <a href="https://cloud.google.com/kubernetes-engine/docs/concepts/persistent-volumes#deployments_vs_statefulsets" rel="nofollow noreferrer">Cloud.google.com: Persistent Volume: Deployments vs statefulsets</a>.</p>
