<p>I am facing issues while mounting pvc on pods with openebs installed on bare-metal kubernetes cluster created with RKE.</p>

<h2>Expected Behavior</h2>

<p>PVC's should be mounted on pods without issues.</p>

<h2>Current Behavior</h2>

<p>Pods unable to mount PVC's:</p>

<pre><code>  Events:
  Type     Reason                  Age                    From                     Message
  ----     ------                  ----                   ----                     -------
  Warning  FailedScheduling        2m9s (x23 over 2m45s)  default-scheduler        pod has unbound PersistentVolumeClaims (repeated 4 times)
  Normal   Scheduled               2m8s                   default-scheduler        Successfully assigned default/minio-deployment-64d7c79464-966jr to 192.168.1.21
  Normal   SuccessfulAttachVolume  2m8s                   attachdetach-controller  AttachVolume.Attach succeeded for volume "pvc-63cf6c92-ec99-11e8-85c9-b06ebfd124ff"
  Warning  FailedMount             84s (x4 over 102s)     kubelet, 192.168.1.21    MountVolume.WaitForAttach failed for volume "pvc-63cf6c92-ec99-11e8-85c9-b06ebfd124ff" : failed to get any path for iscsi disk, last err seen:
iscsi: failed to sendtargets to portal 10.43.227.122:3260 output: iscsiadm: cannot make connection to 10.43.227.122: Connection refused
iscsiadm: cannot make connection to 10.43.227.122: Connection refused
iscsiadm: cannot make connection to 10.43.227.122: Connection refused
iscsiadm: cannot make connection to 10.43.227.122: Connection refused
iscsiadm: cannot make connection to 10.43.227.122: Connection refused
iscsiadm: cannot make connection to 10.43.227.122: Connection refused
iscsiadm: connection login retries (reopen_max) 5 exceeded
iscsiadm: No portals found
, err exit status 21
  Warning  FailedMount  24s (x4 over 80s)  kubelet, 192.168.1.21  MountVolume.WaitForAttach failed for volume "pvc-63cf6c92-ec99-11e8-85c9-b06ebfd124ff" : failed to get any path for iscsi disk, last err seen:
iscsi: failed to attach disk: Error: iscsiadm: Could not login to [iface: default, target: iqn.2016-09.com.openebs.jiva:pvc-63cf6c92-ec99-11e8-85c9-b06ebfd124ff, portal: 10.43.227.122,3260].
iscsiadm: initiator reported error (12 - iSCSI driver not found. Please make sure it is loaded, and retry the operation)
iscsiadm: Could not log into all portals
Logging in to [iface: default, target: iqn.2016-09.com.openebs.jiva:pvc-63cf6c92-ec99-11e8-85c9-b06ebfd124ff, portal: 10.43.227.122,3260] (multiple)
 (exit status 12)
  Warning  FailedMount  2s  kubelet, 192.168.1.21  Unable to mount volumes for pod "minio-deployment-64d7c79464-966jr_default(640263d0-ec99-11e8-85c9-b06ebfd124ff)": timeout expired waiting for volumes to attach or mount for pod "default"/"minio-deployment-64d7c79464-966jr". list of unmounted volumes=[storage]. list of unattached volumes=[storage default-token-9n8pn]
</code></pre>

<h2>Steps to Reproduce</h2>

<ol>
<li>Install openebs with helm.</li>
<li>Create a pvc with storage class as <code>openebs-standalone</code></li>
<li>Create pod and try to mount the PVC.</li>
</ol>

<p><code>kubectl get pvc</code>:</p>

<pre><code>root@an4:/home/rke-k8s# kubectl get pvc
NAME                                      STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS         AGE
docker-private-registry-docker-registry   Bound    pvc-58cf63c1-ec95-11e8-9b5d-2cfda16d3cfd   10Gi       RWO            openebs-standalone   22m
</code></pre>

<p><strong>Update</strong></p>

<p>When I tried the sample minio deployment, here's what I have observed:</p>

<ol>
<li>PVC creation took around 1-2 minutes.</li>
<li>Mounting PVC to the pod took around 1 hour.</li>
<li>Storage class used for this was <code>openebs-standard</code>.</li>
</ol>

<p>Any reason for this? It is on-prem cluster deployment.</p>

Shantanu Deshpande
<p>Well, this issue was documented in the troubleshooting guide - <a href="https://docs.openebs.io/docs/next/tsgiscsi.html" rel="nofollow noreferrer">https://docs.openebs.io/docs/next/tsgiscsi.html</a></p>
