<p>I created a 200G disk with the command <code>gcloud compute disks create --size 200GB my-disk</code></p>

<p>then created a PersistentVolume</p>

<pre>

    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: my-volume
    spec:
      capacity:
        storage: 200Gi
      accessModes:
        - ReadWriteOnce
      gcePersistentDisk:
        pdName: my-disk
        fsType: ext4

</pre>

<p>then created a PersistentVolumeClaim</p>

<pre>

    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: my-claim
    spec:
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: 200Gi

</pre>

<p>then created a StatefulSet and mount the volume to /mnt/disks, which is an existing directory. statefulset.yaml:</p>

<pre>

    apiVersion: apps/v1beta2
    kind: StatefulSet
    metadata:
      name: ...
    spec:
        ...
        spec:
          containers:
          - name: ...
            ...
            volumeMounts:
            - name: my-volume
              mountPath: /mnt/disks
          volumes:
          - name: my-volume
            emptyDir: {}
      volumeClaimTemplates:
      - metadata:
          name: my-claim
        spec:
          accessModes: [ "ReadWriteOnce" ]
          resources:
            requests:
              storage: 200Gi

</pre>

<p>I ran command <code>kubectl get pv</code> and saw that disk was successfully mounted to each instance</p>

<pre>

    NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM                                    STORAGECLASS   REASON    AGE
    my-volume                                  200Gi      RWO            Retain           Available                                                                     19m
    pvc-17c60f45-2e4f-11e8-9b77-42010af0000e   200Gi      RWO            Delete           Bound       default/my-claim-xxx_1   standard                 13m
    pvc-5972c804-2e4e-11e8-9b77-42010af0000e   200Gi      RWO            Delete           Bound       default/my-claim                         standard                 18m
    pvc-61b9daf9-2e4e-11e8-9b77-42010af0000e   200Gi      RWO            Delete           Bound       default/my-claimxxx_0   standard                 18m

</pre>

<p>but when I ssh into an instance and run <code>df -hT</code>, I do not see the mounted volume. below is the output:</p>

<pre>

    Filesystem     Type      Size  Used Avail Use% Mounted on
    /dev/root      ext2      1.2G  447M  774M  37% /
    devtmpfs       devtmpfs  1.9G     0  1.9G   0% /dev
    tmpfs          tmpfs     1.9G     0  1.9G   0% /dev/shm
    tmpfs          tmpfs     1.9G  744K  1.9G   1% /run
    tmpfs          tmpfs     1.9G     0  1.9G   0% /sys/fs/cgroup
    tmpfs          tmpfs     1.9G     0  1.9G   0% /tmp
    tmpfs          tmpfs     256K     0  256K   0% /mnt/disks
    /dev/sda8      ext4       12M   28K   12M   1% /usr/share/oem
    /dev/sda1      ext4       95G  3.5G   91G   4% /mnt/stateful_partition
    tmpfs          tmpfs     1.0M  128K  896K  13% /var/lib/cloud
    overlayfs      overlay   1.0M  148K  876K  15% /etc

</pre>

<p>anyone has any idea?</p>

<p>Also worth mentioning that I'm trying to mount the disk to a docker image which is running in kubernete engine. The pod was created with below commands:</p>

<pre>

    docker build -t gcr.io/xxx .
    gcloud docker -- push gcr.io/xxx
    kubectl create -f statefulset.yaml

</pre>

<p>The instance I sshed into is the one that runs the docker image. I do not see the volume in both instance and the docker container</p>

<p><strong>UPDATE</strong>
I found the volume, I ran <code>df -ahT</code> in the instance, and saw the relevant entries</p>

<pre>

    /dev/sdb       -               -     -     -    - /var/lib/kubelet/plugins/kubernetes.io/gce-pd/mounts/gke-xxx-cluster-c-pvc-61b9daf9-2e4e-11e8-9b77-42010af0000e
    /dev/sdb       -               -     -     -    - /var/lib/kubelet/plugins/kubernetes.io/gce-pd/mounts/gke-xxx-cluster-c-pvc-61b9daf9-2e4e-11e8-9b77-42010af0000e
    /dev/sdb       -               -     -     -    - /home/kubernetes/containerized_mounter/rootfs/var/lib/kubelet/plugins/kubernetes.io/gce-pd/mounts/gke-xxx-cluster-c-pvc-61b9daf9-2e4e-11e8-9b77-42010af0000e
    /dev/sdb       -               -     -     -    - /home/kubernetes/containerized_mounter/rootfs/var/lib/kubelet/plugins/kubernetes.io/gce-pd/mounts/gke-xxx-cluster-c-pvc-61b9daf9-2e4e-11e8-9b77-42010af0000e
    /dev/sdb       -               -     -     -    - /var/lib/kubelet/pods/61bb679b-2e4e-11e8-9b77-42010af0000e/volumes/kubernetes.io~gce-pd/pvc-61b9daf9-2e4e-11e8-9b77-42010af0000e
    /dev/sdb       -               -     -     -    - /var/lib/kubelet/pods/61bb679b-2e4e-11e8-9b77-42010af0000e/volumes/kubernetes.io~gce-pd/pvc-61b9daf9-2e4e-11e8-9b77-42010af0000e
    /dev/sdb       -               -     -     -    - /home/kubernetes/containerized_mounter/rootfs/var/lib/kubelet/pods/61bb679b-2e4e-11e8-9b77-42010af0000e/volumes/kubernetes.io~gce-pd/pvc-61b9daf9-2e4e-11e8-9b77-42010af0000e
    /dev/sdb       -               -     -     -    - /home/kubernetes/containerized_mounter/rootfs/var/lib/kubelet/pods/61bb679b-2e4e-11e8-9b77-42010af0000e/volumes/kubernetes.io~gce-pd/pvc-61b9daf9-2e4e-11e8-9b77-42010af0000e

</pre> 

<p>then I went into the docker container and ran <code>df -ahT</code>, I got</p>

<pre>

    Filesystem     Type     Size  Used Avail Use% Mounted on
    /dev/sda1      ext4      95G  3.5G   91G   4% /mnt/disks

</pre>

<p>Why I'm seeing 95G total size instead of 200G, which is the size of my volume?</p>

<p>More info:
<code>kubectl describe pod</code></p>

<pre>

    Name:           xxx-replicaset-0
    Namespace:      default
    Node:           gke-xxx-cluster-default-pool-5e49501c-nrzt/10.128.0.17
    Start Time:     Fri, 23 Mar 2018 11:40:57 -0400
    Labels:         app=xxx-replicaset
                    controller-revision-hash=xxx-replicaset-755c4f7cff
    Annotations:    kubernetes.io/created-by={"kind":"SerializedReference","apiVersion":"v1","reference":{"kind":"StatefulSet","namespace":"default","name":"xxx-replicaset","uid":"d6c3511f-2eaf-11e8-b14e-42010af0000...
                    kubernetes.io/limit-ranger=LimitRanger plugin set: cpu request for container xxx-deployment
    Status:         Running
    IP:             10.52.4.5
    Created By:     StatefulSet/xxx-replicaset
    Controlled By:  StatefulSet/xxx-replicaset
    Containers:
      xxx-deployment:
        Container ID:   docker://137b3966a14538233ed394a3d0d1501027966b972d8ad821951f53d9eb908615
        Image:          gcr.io/sampeproject/xxxstaging:v1
        Image ID:       docker-pullable://gcr.io/sampeproject/xxxstaging@sha256:a96835c2597cfae3670a609a69196c6cd3d9cc9f2f0edf5b67d0a4afdd772e0b
        Port:           8080/TCP
        State:          Running
          Started:      Fri, 23 Mar 2018 11:42:17 -0400
        Ready:          True
        Restart Count:  0
        Requests:
          cpu:        100m
        Environment:  
        Mounts:
          /mnt/disks from my-volume (rw)
          /var/run/secrets/kubernetes.io/serviceaccount from default-token-hj65g (ro)
    Conditions:
      Type           Status
      Initialized    True
      Ready          True
      PodScheduled   True
    Volumes:
      my-claim:
        Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
        ClaimName:  my-claim-xxx-replicaset-0
        ReadOnly:   false
      my-volume:
        Type:    EmptyDir (a temporary directory that shares a pod's lifetime)
        Medium:
      default-token-hj65g:
        Type:        Secret (a volume populated by a Secret)
        SecretName:  default-token-hj65g
        Optional:    false
    QoS Class:       Burstable
    Node-Selectors:  
    Tolerations:     node.alpha.kubernetes.io/notReady:NoExecute for 300s
                     node.alpha.kubernetes.io/unreachable:NoExecute for 300s
    Events:
      Type     Reason                 Age                From                                                      Message
      ----     ------                 ----               ----                                                      -------
      Warning  FailedScheduling       10m (x4 over 10m)  default-scheduler                                         PersistentVolumeClaim is not bound: "my-claim-xxx-replicaset-0" (repeated 5 times)
      Normal   Scheduled              9m                 default-scheduler                                         Successfully assigned xxx-replicaset-0 to gke-xxx-cluster-default-pool-5e49501c-nrzt
      Normal   SuccessfulMountVolume  9m                 kubelet, gke-xxx-cluster-default-pool-5e49501c-nrzt  MountVolume.SetUp succeeded for volume "my-volume"
      Normal   SuccessfulMountVolume  9m                 kubelet, gke-xxx-cluster-default-pool-5e49501c-nrzt  MountVolume.SetUp succeeded for volume "default-token-hj65g"
      Normal   SuccessfulMountVolume  9m                 kubelet, gke-xxx-cluster-default-pool-5e49501c-nrzt  MountVolume.SetUp succeeded for volume "pvc-902c57c5-2eb0-11e8-b14e-42010af0000e"
      Normal   Pulling                9m                 kubelet, gke-xxx-cluster-default-pool-5e49501c-nrzt  pulling image "gcr.io/sampeproject/xxxstaging:v1"
      Normal   Pulled                 8m                 kubelet, gke-xxx-cluster-default-pool-5e49501c-nrzt  Successfully pulled image "gcr.io/sampeproject/xxxstaging:v1"
      Normal   Created                8m                 kubelet, gke-xxx-cluster-default-pool-5e49501c-nrzt  Created container
      Normal   Started                8m                 kubelet, gke-xxx-cluster-default-pool-5e49501c-nrzt  Started container

</pre>

<p>Seems like it did not mount the correct volume. I ran <code>lsblk</code> in docker container</p>

<pre>
NAME    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT

    sda       8:0    0  100G  0 disk 
    ├─sda1    8:1    0 95.9G  0 part /mnt/disks
    ├─sda2    8:2    0   16M  0 part 
    ├─sda3    8:3    0    2G  0 part 
    ├─sda4    8:4    0   16M  0 part 
    ├─sda5    8:5    0    2G  0 part 
    ├─sda6    8:6    0  512B  0 part 
    ├─sda7    8:7    0  512B  0 part 
    ├─sda8    8:8    0   16M  0 part 
    ├─sda9    8:9    0  512B  0 part 
    ├─sda10   8:10   0  512B  0 part 
    ├─sda11   8:11   0    8M  0 part 
    └─sda12   8:12   0   32M  0 part 
    sdb       8:16   0  200G  0 disk 

</pre>

<p>Why this is happening?</p>

<p>When you use PVCs, K8s manages persistent disks for you.</p>

<p>The exact way how PVs can by defined by provisioner in storage classes. Since you use GKE your default SC uses kubernetes.io/gce-pd provisioner (<a href="https://kubernetes.io/docs/concepts/storage/storage-classes/#gce" rel="nofollow noreferrer">https://kubernetes.io/docs/concepts/storage/storage-classes/#gce</a>).</p>

<p>In other words for each pod new PV is created.</p>

<p>If you would like to use existing disk you can use Volumes instead of PVCs (<a href="https://kubernetes.io/docs/concepts/storage/volumes/#gcepersistentdisk" rel="nofollow noreferrer">https://kubernetes.io/docs/concepts/storage/volumes/#gcepersistentdisk</a>)</p>
