{"action":"opened","issue":{"url":"https://api.github.com/repos/ceph/ceph-docker/issues/437","repository_url":"https://api.github.com/repos/ceph/ceph-docker","labels_url":"https://api.github.com/repos/ceph/ceph-docker/issues/437/labels{/name}","comments_url":"https://api.github.com/repos/ceph/ceph-docker/issues/437/comments","events_url":"https://api.github.com/repos/ceph/ceph-docker/issues/437/events","html_url":"https://github.com/ceph/ceph-docker/issues/437","id":189159277,"number":437,"title":"Can't create OSD pod from k8s example","user":{"login":"danufriev","id":17832983,"avatar_url":"https://avatars.githubusercontent.com/u/17832983?v=3","gravatar_id":"","url":"https://api.github.com/users/danufriev","html_url":"https://github.com/danufriev","followers_url":"https://api.github.com/users/danufriev/followers","following_url":"https://api.github.com/users/danufriev/following{/other_user}","gists_url":"https://api.github.com/users/danufriev/gists{/gist_id}","starred_url":"https://api.github.com/users/danufriev/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/danufriev/subscriptions","organizations_url":"https://api.github.com/users/danufriev/orgs","repos_url":"https://api.github.com/users/danufriev/repos","events_url":"https://api.github.com/users/danufriev/events{/privacy}","received_events_url":"https://api.github.com/users/danufriev/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2016-11-14T16:32:31Z","updated_at":"2016-11-14T16:32:31Z","closed_at":null,"body":"I have built ceph cluster with k8s on ubuntu 16.04 with this tutorial https://github.com/ceph/ceph-docker/tree/master/examples/kubernetes\r\nAll components were up. But I have some issue with OSD pods. I'm getting error \"-1 unable to find any IP address in networks: 10.96.0.0/12\".\r\nAdditional information from my k8s cluster:\r\n```\r\nroot@k8s:~/ceph-docker/examples/kubernetes# kubectl get pods --namespace=ceph\r\nNAME                              READY     STATUS             RESTARTS   AGE\r\nceph-mds-1459951177-j1z78         1/1       Running            3          1h\r\nceph-mon-3227890741-dphzx         1/1       Running            2          1h\r\nceph-mon-3227890741-t6dgh         1/1       Running            0          1h\r\nceph-mon-3227890741-umhy1         1/1       Running            2          1h\r\nceph-mon-check-2664903432-l0jtl   1/1       Running            0          1h\r\nceph-osd-1wdgd                    0/1       CrashLoopBackOff   25         1h\r\nceph-osd-hljci                    0/1       CrashLoopBackOff   25         1h\r\nceph-osd-js6v7                    0/1       CrashLoopBackOff   24         1h\r\nroot@k8s:~/ceph-docker/examples/kubernetes# kubectl get svc --namespace=ceph\r\nNAME       CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE\r\nceph-mon   None         <none>        6789/TCP   1h\r\nroot@k8s:~/ceph-docker/examples/kubernetes# nslookup\r\n> lserver 10.96.0.10\r\nDefault server: 10.96.0.10\r\nAddress: 10.96.0.10#53\r\n> ceph-mon.ceph.svc.cluster.local\r\nServer:\t\t10.96.0.10\r\nAddress:\t10.96.0.10#53\r\n\r\nName:\tceph-mon.ceph.svc.cluster.local\r\nAddress: 10.32.0.3\r\nName:\tceph-mon.ceph.svc.cluster.local\r\nAddress: 10.36.0.2\r\nName:\tceph-mon.ceph.svc.cluster.local\r\nAddress: 10.44.0.2\r\n> \r\nroot@k8s:~/ceph-docker/examples/kubernetes# kubectl logs pods/ceph-osd-1wdgd --namespace=ceph\r\n2016-11-14 16:11:04  /entrypoint.sh: k8s: config is stored as k8s secrets.\r\n2016-11-14 16:11:04  /entrypoint.sh: k8s: does not generate admin key. Use secrets instead.\r\nceph-1: /usr/bin/ceph-osd --cluster ceph -f -d -i 1 --osd-journal /var/lib/ceph/osd/ceph-1/journal -k /var/lib/ceph/osd/ceph-1/keyring\r\n2016-11-14 16:11:04  /entrypoint.sh: SUCCESS\r\nforego  | starting ceph-1.1 on port 5000\r\nceph-1.1 | 2016-11-14 16:11:04.573117 7fd26f734800  0 ceph version 10.2.3 (ecc23778eb545d8dd55e2e4735b53cc93f92e65b), process ceph-osd, pid 30\r\nceph-1.1 | 2016-11-14 16:11:04.574131 7fd26f734800 -1 unable to find any IP address in networks: 10.96.0.0/12\r\n\r\nroot@k8s:~/ceph-docker/examples/kubernetes#\r\n```\r\nAlso I've changed ceph.conf with changing nom_host string, but this not gave me any result:\r\n\r\n```\r\nroot@k8s:~/ceph-docker/examples/kubernetes# cat /root/ceph-docker/examples/kubernetes/generator/ceph.conf \r\n\r\nmon_host = ceph-mon.ceph.svc.cluster.local\r\n\r\n[osd]\r\njournal_size = 100\r\ncluster_network = 10.96.0.0/12\r\npublic_network = 10.96.0.0/12\r\nosd_mkfs_type = xfs\r\nosd_mkfs_options_xfs = -f -i size=2048\r\nosd_mon_heartbeat_interval = 30\r\nosd_max_object_name_len = 256\r\n```\r\nAlso mon status is unhealthy:\r\n```\r\nroot@k8s:~# docker exec c1896e67b162d1105294f2e1f278d1238969c47605ddc5437b935e8d49575072 ceph -s\r\n    cluster 25c00ce5-d03e-4c7c-b7e6-2e570290512c\r\n     health HEALTH_ERR\r\n            80 pgs are stuck inactive for more than 300 seconds\r\n            80 pgs stuck inactive\r\n            no osds\r\n     monmap e1: 3 mons at {ceph-mon-3227890741-dphzx=10.36.0.2:6789/0,ceph-mon-3227890741-t6dgh=10.32.0.3:6789/0,ceph-mon-3227890741-umhy1=10.44.0.2:6789/0}\r\n            election epoch 6, quorum 0,1,2 ceph-mon-3227890741-t6dgh,ceph-mon-3227890741-dphzx,ceph-mon-3227890741-umhy1\r\n      fsmap e2: 0/0/1 up\r\n     osdmap e4: 0 osds: 0 up, 0 in\r\n            flags sortbitwise\r\n      pgmap v5: 80 pgs, 3 pools, 0 bytes data, 0 objects\r\n            0 kB used, 0 kB / 0 kB avail\r\n                  80 creating\r\nroot@k8s:~#\r\n``` \r\nHow to fix this issue?"}}
{"actor":{"display_login":"danufriev"}}