<p>I've been delving into Rook+Ceph for Kubernetes, trying to get it to work under Google Kubernetes Engine, and have hit a brick wall.</p>

<p>Following the documentation, I've run the following commands, and verified that each has had the intended effect:</p>

<pre><code>kubectl create -f common.yaml
kubectl create -f operator.yaml
kubectl create -f cluster.yaml
kubectl create -f filesystem.yaml
</code></pre>

<p>The yaml files are all default as <a href="https://github.com/rook/rook/tree/release-1.0/cluster/examples/kubernetes/ceph" rel="nofollow noreferrer">provided by Rook</a>, with the exception of <code>operator.yaml</code>, wherein I've added the following environment variable, as per <a href="https://rook.io/docs/rook/v1.0/flexvolume.html" rel="nofollow noreferrer">the GKE docs</a>:</p>

<pre><code>- name: FLEXVOLUME_DIR_PATH
  value: "/home/kubernetes/flexvolume"
</code></pre>

<p>I am unsure where to go from here, however. Their documentation has a sample file for <a href="https://rook.io/docs/rook/v1.0/ceph-filesystem.html" rel="nofollow noreferrer">creating a registry</a>, which leads down a rabbit-hole of instructions which appear unrelated to what I'm trying to achieve.</p>

<p>Essentially, I want to be able to mount Ceph shared file storage as a volume from a regular ol' container image. I've tried the following yaml with no success:</p>

<pre><code>apiVersion: v1
kind: Pod
metadata:
  name: ubuntu
  namespace: default
spec:
  containers:
  - name: ubuntu
    image: ubuntu:bionic
    command:
      - sleep
      - "3600"
    imagePullPolicy: IfNotPresent
    volumeMounts:
    - name: image-store
      mountPath: /mnt/shared
  volumes:
  - name: image-store
    flexVolume:
      driver: ceph.rook.io/rook
      fsType: ceph
      options:
        fsName: myfs
        clusterNamespace: rook-ceph
  restartPolicy: Always
</code></pre>

<p>In my fairly limited knowledge, I can't see why this wouldn't work, but am getting the following output when doing <code>kubectl describe pods ubuntu</code>:</p>

<pre><code>Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/10ee80d1-8873-11e9-8f45-42010a8a01d8/volumes/ceph.rook.io~rook/image-store --scope -- mount -t ceph -o name=admin,secret=AQDAM/lcQFKHARAAh/O27Wl+iRKRzREsMML+4g==,mds_namespace=myfs 10.8.3.134:6789,10.8.13.226:6789,10.8.5.39:6789:/ /var/lib/kubelet/pods/10ee80d1-8873-11e9-8f45-42010a8a01d8/volumes/ceph.rook.io~rook/image-store
Output: Running scope as unit: run-reb61186d1ff64e6e846a200580aa5395.scope
mount: /var/lib/kubelet/pods/10ee80d1-8873-11e9-8f45-42010a8a01d8/volumes/ceph.rook.io~rook/image-store: special device 10.8.3.134:6789,10.8.13.226:6789,10.8.5.39:6789:/ does not exist.
  Warning  FailedMount  51s (x2 over 2m53s)  kubelet, gke-kubey-cluster-default-pool-6856b374-nb0c  (combined from similar events): MountVolume.SetUp failed for volume "image-store" : mount command failed, status: Failure, reason: failed to mount filesystem myfs to /var/lib/kubelet/pods/10ee80d1-8873-11e9-8f45-42010a8a01d8/volumes/ceph.rook.io~rook/image-store with monitor 10.8.3.134:6789,10.8.13.226:6789,10.8.5.39:6789:/ and options [name=admin secret=AQDAM/lcQFKHARAAh/O27Wl+iRKRzREsMML+4g== mds_namespace=myfs]: mount failed: exit status 32
</code></pre>

<p>Is there an example of how to achieve such a thing somewhere in the wild? </p>

Murray
<p>Make sure you fillfull every single prerequisities for Rook setup: <a href="https://github.com/rook/rook/blob/master/Documentation/psp.md" rel="nofollow noreferrer">rook_setup</a>.</p>

<p>Then try to install :</p>

<pre><code>nfs-common
</code></pre>

<p>on all your nodes.</p>
