{"action":"created","issue":{"url":"https://api.github.com/repos/kubernetes/cloud-provider-openstack/issues/249","repository_url":"https://api.github.com/repos/kubernetes/cloud-provider-openstack","labels_url":"https://api.github.com/repos/kubernetes/cloud-provider-openstack/issues/249/labels{/name}","comments_url":"https://api.github.com/repos/kubernetes/cloud-provider-openstack/issues/249/comments","events_url":"https://api.github.com/repos/kubernetes/cloud-provider-openstack/issues/249/events","html_url":"https://github.com/kubernetes/cloud-provider-openstack/issues/249","id":346898051,"node_id":"MDU6SXNzdWUzNDY4OTgwNTE=","number":249,"title":"Unable to mount volumes for pod with the pvc created by cinder-provisioner","user":{"login":"npu21","id":11768607,"node_id":"MDQ6VXNlcjExNzY4NjA3","avatar_url":"https://avatars3.githubusercontent.com/u/11768607?v=4","gravatar_id":"","url":"https://api.github.com/users/npu21","html_url":"https://github.com/npu21","followers_url":"https://api.github.com/users/npu21/followers","following_url":"https://api.github.com/users/npu21/following{/other_user}","gists_url":"https://api.github.com/users/npu21/gists{/gist_id}","starred_url":"https://api.github.com/users/npu21/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/npu21/subscriptions","organizations_url":"https://api.github.com/users/npu21/orgs","repos_url":"https://api.github.com/users/npu21/repos","events_url":"https://api.github.com/users/npu21/events{/privacy}","received_events_url":"https://api.github.com/users/npu21/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":7,"created_at":"2018-08-02T07:49:30Z","updated_at":"2018-08-08T08:06:22Z","closed_at":null,"author_association":"NONE","body":"<!-- This form is for bug reports and feature requests! -->\r\n\r\n**Is this a BUG REPORT or FEATURE REQUEST?**:\r\n\r\n> Uncomment only one, leave it on its own line: \r\n>\r\n> /kind bug\r\n\r\n\r\n**What happened**:\r\nI create a deployment like this\r\n```\r\napiVersion: apps/v1beta2\r\nkind: Deployment          \r\nmetadata:\r\n  name: mysql-sonar\r\nspec:\r\n  replicas: 1                          \r\n  selector:\r\n    matchLabels:\r\n      app: mysql-sonar                      \r\n  template:\r\n    metadata:\r\n      labels:\r\n        app: mysql-sonar\r\n    spec:\r\n      containers:                       \r\n      - name: mysql-sonar\r\n        image: harbor.iop.com:5000/library/iop/mysql:5.7.22\r\n        args:\r\n          - \"--ignore-db-dir=lost+found\"      \r\n        ports:\r\n        - containerPort: 3306           \r\n        env:\r\n        - name: MYSQL_ROOT_PASSWORD\r\n          value: 123456XXX\r\n        volumeMounts:\r\n        - name: mysql-sonar\r\n          mountPath: /var/mysql\r\n      volumes:\r\n      - name: mysql-sonar\r\n        persistentVolumeClaim:\r\n          claimName: mysql-sonar-pvc\r\n```\r\nthe pvc mysql-sonar-pvc is create by cinder-provisioner\r\n```\r\napiVersion: v1\r\nkind: PersistentVolumeClaim\r\nmetadata:\r\n  name: mysql-sonar-pvc\r\nspec:\r\n  accessModes:\r\n  - ReadWriteOnce\r\n  resources:\r\n    requests:\r\n      storage: 1Gi\r\n  storageClassName: cinder-standard-iops\r\n```\r\nand it's bound\r\nmysql-sonar-pvc                              Bound     pvc-4c853f30-9625-11e8-bfa3-0050568860dd   1Gi        RWO            cinder-standard-iops   13m\r\nI can also see it's created successfully in cinder api\r\n\r\n> Name:            pvc-4c853f30-9625-11e8-bfa3-0050568860dd\r\n> Labels:          <none>\r\n> Annotations:     cinderVolumeId=2ba19e9b-9430-44d2-8198-dbc6ca28acd5\r\n>                  pv.kubernetes.io/provisioned-by=openstack.org/standalone-cinder\r\n>                  standaloneCinderProvisionerIdentity=openstack.org/standalone-cinder\r\n> StorageClass:    cinder-standard-iops\r\n> Status:          Bound\r\n> Claim:           default/mysql-sonar-pvc\r\n> Reclaim Policy:  Delete\r\n> Access Modes:    RWO\r\n> Capacity:        1Gi\r\n> Message:         \r\n> Source:\r\n>     Type:          RBD (a Rados Block Device mount on the host that shares a pod's lifetime)\r\n>     CephMonitors:  [192.168.4.181:6789 192.168.4.182:6789 192.168.4.183:6789]\r\n>     RBDImage:      volume-2ba19e9b-9430-44d2-8198-dbc6ca28acd5\r\n>     FSType:        \r\n>     RBDPool:       volumes\r\n>     RadosUser:     cinder\r\n>     Keyring:       /etc/ceph/keyring\r\n>     SecretRef:     &{cinder-standard-iops-cephx-secret }\r\n>     ReadOnly:      false\r\n> Events:            <none>\r\n\r\nhere's the secret \r\n```\r\nName:         cinder-standard-iops-cephx-secret\r\nNamespace:    default\r\nLabels:       <none>\r\nAnnotations:  <none>\r\n\r\nType:  Opaque\r\n\r\nData\r\n====\r\nkey:  41 bytes\r\n```\r\n**What you expected to happen**:\r\nthe deployment are expected run successfully,however it failed at this step\r\n![image](https://user-images.githubusercontent.com/11768607/43570026-e4783b62-966b-11e8-92f6-692676749cd1.png)\r\nand here's the more detail kubernetes log\r\nAug 01 21:08:50 master2 kubelet[2490]: I0801 21:08:50.540947    2490 reconciler.go:217] operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-ea7ba804-955b-11e8-99c3-0050568860dd\" (UniqueName: \"kubernetes.io/rbd/[192.168.4.181:6789 192.168.4.182:6789 192.168.4.183:6789]:volume-98adc6e3-c108-4f62-9cb3-91aebab2094d\") pod \"mysql-sonar-7f75f65b64-sv5pn\" (UID: \"5643209b-95f0-11e8-905f-005056885e00\")\r\nAug 01 21:08:50 master2 kubelet[2490]: I0801 21:08:50.546307    2490 operation_generator.go:1121] Controller attach succeeded for volume \"pvc-ea7ba804-955b-11e8-99c3-0050568860dd\" (UniqueName: \"kubernetes.io/rbd/[192.168.4.181:6789 192.168.4.182:6789 192.168.4.183:6789]:volume-98adc6e3-c108-4f62-9cb3-91aebab2094d\") pod \"mysql-sonar-7f75f65b64-sv5pn\" (UID: \"5643209b-95f0-11e8-905f-005056885e00\") device path: \"\"\r\nAug 01 21:08:50 master2 kubelet[2490]: I0801 21:08:50.647216    2490 reconciler.go:262] operationExecutor.MountVolume started for volume \"pvc-ea7ba804-955b-11e8-99c3-0050568860dd\" (UniqueName: \"kubernetes.io/rbd/[192.168.4.181:6789 192.168.4.182:6789 192.168.4.183:6789]:volume-98adc6e3-c108-4f62-9cb3-91aebab2094d\") pod \"mysql-sonar-7f75f65b64-sv5pn\" (UID: \"5643209b-95f0-11e8-905f-005056885e00\")\r\nAug 01 21:08:50 master2 kubelet[2490]: I0801 21:08:50.647296    2490 operation_generator.go:447] MountVolume.WaitForAttach entering for volume \"pvc-ea7ba804-955b-11e8-99c3-0050568860dd\" (UniqueName: \"kubernetes.io/rbd/[192.168.4.181:6789 192.168.4.182:6789 192.168.4.183:6789]:volume-98adc6e3-c108-4f62-9cb3-91aebab2094d\") pod \"mysql-sonar-7f75f65b64-sv5pn\" (UID: \"5643209b-95f0-11e8-905f-005056885e00\") DevicePath \"\"\r\nAug 01 21:08:51 master2 kubelet[2490]: E0801 21:08:51.367069    2490 kubelet.go:1630] Unable to mount volumes for pod \"mysql-sonar-7f75f65b64-sv5pn_default(5643209b-95f0-11e8-905f-005056885e00)\": timeout expired waiting for volumes to attach/mount for pod \"default\"/\"mysql-sonar-7f75f65b64-sv5pn\". list of unattached/unmounted volumes=[mysql-sonar]; skipping pod\r\nAug 01 21:08:51 master2 kubelet[2490]: E0801 21:08:51.367159    2490 pod_workers.go:186] Error syncing pod 5643209b-95f0-11e8-905f-005056885e00 (\"mysql-sonar-7f75f65b64-sv5pn_default(5643209b-95f0-11e8-905f-005056885e00)\"), skipping: timeout expired waiting for volumes to attach/mount for pod \"default\"/\"mysql-sonar-7f75f65b64-sv5pn\". list of unattached/unmounted volumes=[mysql-sonar]\r\n\r\nAny help will be appreciated very much\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/kubernetes/cloud-provider-openstack/issues/comments/411323183","html_url":"https://github.com/kubernetes/cloud-provider-openstack/issues/249#issuecomment-411323183","issue_url":"https://api.github.com/repos/kubernetes/cloud-provider-openstack/issues/249","id":411323183,"node_id":"MDEyOklzc3VlQ29tbWVudDQxMTMyMzE4Mw==","user":{"login":"embik","id":10295525,"node_id":"MDQ6VXNlcjEwMjk1NTI1","avatar_url":"https://avatars1.githubusercontent.com/u/10295525?v=4","gravatar_id":"","url":"https://api.github.com/users/embik","html_url":"https://github.com/embik","followers_url":"https://api.github.com/users/embik/followers","following_url":"https://api.github.com/users/embik/following{/other_user}","gists_url":"https://api.github.com/users/embik/gists{/gist_id}","starred_url":"https://api.github.com/users/embik/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/embik/subscriptions","organizations_url":"https://api.github.com/users/embik/orgs","repos_url":"https://api.github.com/users/embik/repos","events_url":"https://api.github.com/users/embik/events{/privacy}","received_events_url":"https://api.github.com/users/embik/received_events","type":"User","site_admin":false},"created_at":"2018-08-08T08:06:22Z","updated_at":"2018-08-08T08:06:22Z","author_association":"NONE","body":"Would you mind sharing how you deployed the cinder-standalone-provisioner? I'm not sure if it helps with this issue, but I'm not seeing the `*-cephx-secret` secret being created so I'm wondering what's different to your setup."}}
{"actor":{"display_login":"embik"}}