{"action":"created","issue":{"url":"https://api.github.com/repos/01org/ciao/issues/924","repository_url":"https://api.github.com/repos/01org/ciao","labels_url":"https://api.github.com/repos/01org/ciao/issues/924/labels{/name}","comments_url":"https://api.github.com/repos/01org/ciao/issues/924/comments","events_url":"https://api.github.com/repos/01org/ciao/issues/924/events","html_url":"https://github.com/01org/ciao/issues/924","id":193186313,"number":924,"title":"ciao-deployment : not able to launch instances in clearlinux 12000","user":{"login":"arturo-garcia-garcia","id":16679224,"avatar_url":"https://avatars.githubusercontent.com/u/16679224?v=3","gravatar_id":"","url":"https://api.github.com/users/arturo-garcia-garcia","html_url":"https://github.com/arturo-garcia-garcia","followers_url":"https://api.github.com/users/arturo-garcia-garcia/followers","following_url":"https://api.github.com/users/arturo-garcia-garcia/following{/other_user}","gists_url":"https://api.github.com/users/arturo-garcia-garcia/gists{/gist_id}","starred_url":"https://api.github.com/users/arturo-garcia-garcia/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/arturo-garcia-garcia/subscriptions","organizations_url":"https://api.github.com/users/arturo-garcia-garcia/orgs","repos_url":"https://api.github.com/users/arturo-garcia-garcia/repos","events_url":"https://api.github.com/users/arturo-garcia-garcia/events{/privacy}","received_events_url":"https://api.github.com/users/arturo-garcia-garcia/received_events","type":"User","site_admin":false},"labels":[{"id":477597796,"url":"https://api.github.com/repos/01org/ciao/labels/ciao-deployment","name":"ciao-deployment","color":"1d76db","default":false},{"id":347039388,"url":"https://api.github.com/repos/01org/ciao/labels/question","name":"question","color":"cc317c","default":true}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2016-12-02T18:34:54Z","updated_at":"2016-12-02T20:48:32Z","closed_at":null,"body":"Hi, I try to launch an instance of clearlinux on ciao, but it returns an Internal server error\r\n\r\nI have a 4 devices cluster, 1 controller, 1 network and 2 computes,  I'm running clearlinux release 12000 in all of them and using ciao release 76\r\n\r\nhere are my workloads \r\n\r\n```\r\nroot@controller.example.com ~ # ciao-cli workload list\r\nWorkload 1\r\n\tName: Docker Iperf\r\n\tUUID:ab68111c-03a6-11e6-87de-001320fb6e31\r\n\tImage UUID: b5e696b8-03a6-11e6-a424-001320fb6e31\r\n\tCPUs: 2\r\n\tMemory: 128 MB\r\nWorkload 2\r\n\tName: Fedora 23 Cloud\r\n\tUUID:79034317-3beb-447e-987d-4e310a8cf410\r\n\tImage UUID: \r\n\tCPUs: 2\r\n\tMemory: 128 MB\r\nWorkload 3\r\n\tName: Clear Cloud\r\n\tUUID:e35ed972-c46c-4aad-a1e7-ef103ae079a2\r\n\tImage UUID: \r\n\tCPUs: 2\r\n\tMemory: 128 MB\r\nWorkload 4\r\n\tName: Docker Ubuntu latest\r\n\tUUID:ca957444-fa46-11e5-94f9-38607786d9ec\r\n\tImage UUID: fa7d86d8-fa46-11e5-8493-38607786d9ec\r\n\tCPUs: 2\r\n\tMemory: 128 MB\r\n```\r\nHere are my images (looks like they didn't end the creation process)\r\n\r\n```\r\nroot@controller.example.com ~/BAT/cloud/ciao/_DeploymentAndDistroPackaging/ansible # ciao-cli image list\r\nImage #1\r\n        Name             [-protected]\r\n        Size             [0 bytes]\r\n        UUID             [4acafe8d-3889-49a4-868b-aeffa2060d35]\r\n        Status           [queued]\r\n        Owner            []\r\n        Disk format      []\r\n        Container format []\r\n        Minimal disk     [0 GB]\r\n        Minimal memory   [0 MB]\r\n        Checksum         []\r\n        Visibility       [public]\r\n        Protected        [false]\r\n        CreatedDate      [2016-12-01T18:35:38.860076661Z]\r\n        LastUpdate       []\r\n        Tags             [[]]\r\n        File             [/v2/images/4acafe8d-3889-49a4-868b-aeffa2060d35/file]\r\n        Schema           [/v2/schemas/image]\r\n\r\nImage #2\r\n        Name             [-protected]\r\n        Size             [0 bytes]\r\n        UUID             [a0154990-54b4-4a96-9f72-8be13732e1de]\r\n        Status           [queued]\r\n        Owner            []\r\n        Disk format      []\r\n        Container format []\r\n        Minimal disk     [0 GB]\r\n        Minimal memory   [0 MB]\r\n        Checksum         []\r\n        Visibility       [public]\r\n        Protected        [false]\r\n        CreatedDate      [2016-12-02T16:45:08.713877026Z]\r\n        LastUpdate       []\r\n        Tags             [[]]\r\n        File             [/v2/images/a0154990-54b4-4a96-9f72-8be13732e1de/file]\r\n        Schema           [/v2/schemas/image]\r\n\r\nImage #3\r\n        Name             [-protected]\r\n        Size             [0 bytes]\r\n        UUID             [a914f4d8-08e8-4a1a-ae9d-76b398b7906b]\r\n        Status           [queued]\r\n        Owner            []\r\n        Disk format      []\r\n        Container format []\r\n        Minimal disk     [0 GB]\r\n        Minimal memory   [0 MB]\r\n        Checksum         []\r\n        Visibility       [public]\r\n        Protected        [false]\r\n        CreatedDate      [2016-12-01T22:18:50.422954057Z]\r\n        LastUpdate       []\r\n        Tags             [[]]\r\n        File             [/v2/images/a914f4d8-08e8-4a1a-ae9d-76b398b7906b/file]\r\n        Schema           [/v2/schemas/image]\r\n\r\n```\r\nmy ceph cluster is up and running\r\n\r\n```\r\nroot@controller.example.com ~ # ceph -s\r\n    cluster 63ffca93-f931-4b1c-8079-831dd7866fd0\r\n     health HEALTH_OK\r\n     monmap e1: 1 mons at {example=192.168.10.10:6789/0}\r\n            election epoch 3, quorum 0 example\r\n      fsmap e5: 1/1/1 up {0=example=up:active}\r\n     osdmap e12: 6 osds: 6 up, 6 in\r\n            flags sortbitwise\r\n      pgmap v23: 320 pgs, 3 pools, 2068 bytes data, 21 objects\r\n            30924 MB used, 149 GB / 179 GB avail\r\n                 320 active+clean\r\n```\r\n\r\nand keystone is up and responding\r\n\r\n```\r\nroot@controller.example.com ~/cloud/ciao/_DeploymentAndDistroPackaging/ansible # openstack user list\r\n+----------------------------------+-------+\r\n| ID                               | Name  |\r\n+----------------------------------+-------+\r\n| 754214bf7ace48a3b387215157b09ce1 | admin |\r\n| 7feb972abe5648f8bf7d08f08cc3ccb8 | demo  |\r\n| a10720b1d2234548b31325e8e94b0b4c | ciao  |\r\n+----------------------------------+-------+\r\n``` \r\n\r\nThis is the output when try to launch and instance\r\n```\r\nroot@controller.example.com ~ # ciao-cli instance add -workload e35ed972-c46c-4aad-a1e7-ef103ae079a2\r\nF1202 18:02:11.979436   12595 instance.go:129] ciao-cli FATAL: HTTP Error [500] for [POST https://controller.example.com:8774/v2.1/7807f7d40a2d46b8a7e1dacd19eca780/servers]: {\"error\":{\"code\":500,\"name\":\"Internal Server Error\",\"message\":\"Unable to Launch Tenant CNCI\"}}\r\nnull\r\ngoroutine 1 [running]:\r\ngithub.com/01org/ciao/vendor/github.com/golang/glog.stacks(0x90ff00, 0xc400000000, 0x10d, 0x15d)\r\n\t/builddir/build/BUILD/ciao-76/src/github.com/01org/ciao/vendor/github.com/golang/glog/glog.go:769 +0xa5\r\ngithub.com/01org/ciao/vendor/github.com/golang/glog.(*loggingT).output(0x8f3b60, 0xc400000003, 0xc4203a23c0, 0x8cec32, 0xb, 0x81, 0x0)\r\n\t/builddir/build/BUILD/ciao-76/src/github.com/01org/ciao/vendor/github.com/golang/glog/glog.go:720 +0x357\r\ngithub.com/01org/ciao/vendor/github.com/golang/glog.(*loggingT).printDepth(0x8f3b60, 0xc400000003, 0x1, 0xc4200f17d8, 0x1, 0x1)\r\n\t/builddir/build/BUILD/ciao-76/src/github.com/01org/ciao/vendor/github.com/golang/glog/glog.go:646 +0x126\r\ngithub.com/01org/ciao/vendor/github.com/golang/glog.FatalDepth(0x1, 0xc4200f17d8, 0x1, 0x1)\r\n\t/builddir/build/BUILD/ciao-76/src/github.com/01org/ciao/vendor/github.com/golang/glog/glog.go:1134 +0x5d\r\nmain.fatalf(0xc4202581a0, 0xcd, 0x0, 0x0, 0x0)\r\n\t/builddir/build/BUILD/ciao-76/src/github.com/01org/ciao/ciao-cli/main.go:132 +0x126\r\nmain.(*instanceAddCommand).run(0xc42009e1b0, 0xc42006e090, 0x0, 0x0, 0xc42006e090, 0x0)\r\n\t/builddir/build/BUILD/ciao-76/src/github.com/01org/ciao/ciao-cli/instance.go:129 +0x5e5\r\nmain.(*command).run(0x8f2070, 0xc42006e090, 0x0, 0x0, 0xc4200a8cd8, 0x0)\r\n\t/builddir/build/BUILD/ciao-76/src/github.com/01org/ciao/ciao-cli/main.go:58 +0x15b\r\nmain.main()\r\n\t/builddir/build/BUILD/ciao-76/src/github.com/01org/ciao/ciao-cli/main.go:466 +0xf7\r\nroot@controller.example.com ~ # \r\n```\r\n\r\nSame error with all the workloads\r\n\r\nThis is the error on the controller node\r\n```\r\nDec 02 18:19:18 controller.example.com ciao-controller[11032]: I1202 18:19:18.417960   11032 identity.go:255] Token validation for [4d5be4395de24472abe48625779cbbdf]\r\nDec 02 18:19:18 controller.example.com ciao-controller[11032]: I1202 18:19:18.418925   11032 identity.go:255] Token validation for [4d5be4395de24472abe48625779cbbdf]\r\nDec 02 18:19:18 controller.example.com ciao-controller[11032]: I1202 18:19:18.431740   11032 sqlite3db.go:683] exec: INSERT or IGNORE into tenants VALUES ('4d5be4395de24472abe48625779cbbdf','','','02:2d:d8:df:b1:c2','');\r\nDec 02 18:19:18 controller.example.com ciao-controller[11032]: I1202 18:19:18.431758   11032 api.go:398] HTTP request [\"GET https://controller.example.com:8774/v2.1/4d5be4395de24472abe48625779cbbdf/servers/detail?limit=10&offset=0 HTTP/1.1\\r\\nConn$\r\nction: close\\r\\nConnection: close\\r\\nContent-Type: application/json\\r\\nX-Auth-Token: 991c0dbee1b147e5b0c51f4696a17c32\\r\\nX-Service-Token: 991c0dbee1b147e5b0c51f4696a17c32\\r\\n\\r\\n\"]\r\nDec 02 18:19:18 controller.example.com ciao-controller[11032]: I1202 18:19:18.431798   11032 api.go:356] Next page marker [] limit [10] offset [0]\r\nDec 02 18:19:18 controller.example.com ciao-controller[11032]: I1202 18:19:18.431807   11032 api.go:318] Get servers limit [10] offset [0]\r\nDec 02 18:19:18 controller.example.com ciao-controller[11032]: I1202 18:19:18.446637   11032 identity.go:255] Token validation for [4d5be4395de24472abe48625779cbbdf]\r\nDec 02 18:19:18 controller.example.com ciao-controller[11032]: I1202 18:19:18.460219   11032 api.go:398] HTTP request [\"GET https://controller.example.com:8774/v2.1/4d5be4395de24472abe48625779cbbdf/servers/detail HTTP/1.1\\r\\nConnection: close\\r\\nC$\r\nnnection: close\\r\\nContent-Type: application/json\\r\\nX-Auth-Token: 991c0dbee1b147e5b0c51f4696a17c32\\r\\nX-Service-Token: 991c0dbee1b147e5b0c51f4696a17c32\\r\\n\\r\\n\"]\r\nDec 02 18:19:18 controller.example.com ciao-controller[11032]: I1202 18:19:18.460252   11032 api.go:356] Next page marker [] limit [0] offset [0]\r\nDec 02 18:19:18 controller.example.com ciao-controller[11032]: I1202 18:19:18.460261   11032 api.go:318] Get servers limit [0] offset [0]\r\nDec 02 18:19:18 controller.example.com ciao-controller[11032]: E1202 18:19:18.466232   11032 instance.go:193] Unable to get block device for image: Error when running: [rbd --id admin clone 4e16e743-265a-4bf2-9fd1-57ada0b28904@ciao-image 8a1b$\r\nd04-1a92-4ed4-9905-6422dcbe9726]: exit status 2\r\nDec 02 18:19:18 controller.example.com ciao-controller[11032]: I1202 18:19:18.466254   11032 command.go:150] error newInstance\r\nDec 02 18:19:18 controller.example.com ciao-controller[11032]: 2016/12/02 18:19:18 http: multiple response.WriteHeader calls\r\nDec 02 18:19:19 controller.example.com ciao-controller[11032]: I1202 18:19:19.559132   11032 identity.go:255] Token validation for [4d5be4395de24472abe48625779cbbdf]\r\nDec 02 18:19:19 controller.example.com ciao-controller[11032]: I1202 18:19:19.571182   11032 api.go:398] HTTP request [\"GET https://controller.example.com:8774/v2.1/4d5be4395de24472abe48625779cbbdf/servers/detail? HTTP/1.1\\r\\nConnection: close\\r\\n$\r\nonnection: close\\r\\nContent-Type: application/json\\r\\nX-Auth-Token: 991c0dbee1b147e5b0c51f4696a17c32\\r\\nX-Service-Token: 991c0dbee1b147e5b0c51f4696a17c32\\r\\n\\r\\n\"]\r\nDec 02 18:19:19 controller.example.com ciao-controller[11032]: I1202 18:19:19.571210   11032 api.go:356] Next page marker [] limit [0] offset [0]\r\nDec 02 18:19:19 controller.example.com ciao-controller[11032]: I1202 18:19:19.571217   11032 api.go:318] Get servers limit [0] offset [0]\r\n```\r\n"},"comment":{"url":"https://api.github.com/repos/01org/ciao/issues/comments/264558801","html_url":"https://github.com/01org/ciao/issues/924#issuecomment-264558801","issue_url":"https://api.github.com/repos/01org/ciao/issues/924","id":264558801,"user":{"login":"arturo-garcia-garcia","id":16679224,"avatar_url":"https://avatars.githubusercontent.com/u/16679224?v=3","gravatar_id":"","url":"https://api.github.com/users/arturo-garcia-garcia","html_url":"https://github.com/arturo-garcia-garcia","followers_url":"https://api.github.com/users/arturo-garcia-garcia/followers","following_url":"https://api.github.com/users/arturo-garcia-garcia/following{/other_user}","gists_url":"https://api.github.com/users/arturo-garcia-garcia/gists{/gist_id}","starred_url":"https://api.github.com/users/arturo-garcia-garcia/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/arturo-garcia-garcia/subscriptions","organizations_url":"https://api.github.com/users/arturo-garcia-garcia/orgs","repos_url":"https://api.github.com/users/arturo-garcia-garcia/repos","events_url":"https://api.github.com/users/arturo-garcia-garcia/events{/privacy}","received_events_url":"https://api.github.com/users/arturo-garcia-garcia/received_events","type":"User","site_admin":false},"created_at":"2016-12-02T20:48:32Z","updated_at":"2016-12-02T20:48:32Z","body":"Hi @albertomurillo here is the output of the command\r\n```\r\nroot@controller.example.com ~ # rbd --id admin clone 4e16e743-265a-4bf2-9fd1-57ada0b28904@ciao-image 8a1bd04-1a92-4ed4-9905-6422dcbe9726\r\nrbd: clone error: (2) No such file or directory2016-12-02 20:43:42.371482 7f21823b0100 -1 librbd: error opening parent image: (2) No such file or directory\r\n\r\n\r\n```\r\n\r\nThe ceph cluster have 3 osds with 2 osds disks  of 30gb each"}}
{"actor":{"display_login":"arturo-garcia-garcia"}}