{"action":"created","issue":{"url":"https://api.github.com/repos/coreos/etcd/issues/6870","repository_url":"https://api.github.com/repos/coreos/etcd","labels_url":"https://api.github.com/repos/coreos/etcd/issues/6870/labels{/name}","comments_url":"https://api.github.com/repos/coreos/etcd/issues/6870/comments","events_url":"https://api.github.com/repos/coreos/etcd/issues/6870/events","html_url":"https://github.com/coreos/etcd/issues/6870","id":190321072,"number":6870,"title":"etcd cluster unhealthy state in ubuntu 16.04","user":{"login":"Vaidyanath","id":5064190,"avatar_url":"https://avatars.githubusercontent.com/u/5064190?v=3","gravatar_id":"","url":"https://api.github.com/users/Vaidyanath","html_url":"https://github.com/Vaidyanath","followers_url":"https://api.github.com/users/Vaidyanath/followers","following_url":"https://api.github.com/users/Vaidyanath/following{/other_user}","gists_url":"https://api.github.com/users/Vaidyanath/gists{/gist_id}","starred_url":"https://api.github.com/users/Vaidyanath/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Vaidyanath/subscriptions","organizations_url":"https://api.github.com/users/Vaidyanath/orgs","repos_url":"https://api.github.com/users/Vaidyanath/repos","events_url":"https://api.github.com/users/Vaidyanath/events{/privacy}","received_events_url":"https://api.github.com/users/Vaidyanath/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2016-11-18T13:37:52Z","updated_at":"2016-11-18T18:04:43Z","closed_at":null,"body":"# \r\nI have created a etcd cluster with secure settings with 3 nodes in ubuntu 16.04\r\nThe services are active and running but the cluster health is unhealthy.\r\nHere is the details of the nodes and the service file details.\r\n\r\n```\r\netcdctl --ca-file=/etc/etcd/ca.pem cluster-health\r\nfailed to check the health of member 60650925d4cae038 on https://192.168.19.171:2379: Get https://192.168.19.171:2379/health: Forbidden\r\nmember 60650925d4cae038 is unreachable: [https://192.168.19.171:2379] are all unreachable\r\nfailed to check the health of member dbce2c420d255724 on https://192.168.19.172:2379: Get https://192.168.19.172:2379/health: Forbidden\r\nmember dbce2c420d255724 is unreachable: [https://192.168.19.172:2379] are all unreachable\r\nfailed to check the health of member f187f63d7f7c60cb on https://192.168.19.173:2379: Get https://192.168.19.173:2379/health: Forbidden\r\nmember f187f63d7f7c60cb is unreachable: [https://192.168.19.173:2379] are all unreachable\r\ncluster is unhealthy\r\n```\r\n\r\nNode-1 Service file\r\n\r\n```\r\n[Unit]\r\nDescription=etcd\r\nDocumentation=https://github.com/coreos\r\n\r\n[Service]\r\nType=notify\r\nExecStart=/usr/bin/etcd --name K8SNode-Master01 \\\r\n  --cert-file=/etc/etcd/kubernetes.pem \\\r\n  --key-file=/etc/etcd/kubernetes-key.pem \\\r\n  --peer-cert-file=/etc/etcd/kubernetes.pem \\\r\n  --peer-key-file=/etc/etcd/kubernetes-key.pem \\\r\n  --trusted-ca-file=/etc/etcd/ca.pem \\\r\n  --peer-trusted-ca-file=/etc/etcd/ca.pem \\\r\n  --initial-advertise-peer-urls https://192.168.19.171:2380 \\\r\n  --listen-peer-urls https://192.168.19.171:2380 \\\r\n  --listen-client-urls https://192.168.19.171:2379,http://127.0.0.1:2379 \\\r\n  --advertise-client-urls https://192.168.19.171:2379 \\\r\n  --initial-cluster-token k8scluster \\\r\n  --initial-cluster K8SNode-Master01=https://192.168.19.171:2380,K8SNode-Master02=https://192.168.19.172:2380,K8SNode-Master03=https://192.168.19.173:2380 \\\r\n  --initial-cluster-state new \\\r\n  --data-dir=/var/lib/etcd\r\nRestart=on-failure\r\nRestartSec=5\r\n\r\n[Install]\r\nWantedBy=multi-user.target\r\n```\r\n\r\nNode-2 Service file\r\n\r\n```\r\n[Unit]\r\nDescription=etcd\r\nDocumentation=https://github.com/coreos\r\n\r\n[Service]\r\nType=notify\r\nExecStart=/usr/bin/etcd --name K8SNode-Master02 \\\r\n  --cert-file=/etc/etcd/kubernetes.pem \\\r\n  --key-file=/etc/etcd/kubernetes-key.pem \\\r\n  --peer-cert-file=/etc/etcd/kubernetes.pem \\\r\n  --peer-key-file=/etc/etcd/kubernetes-key.pem \\\r\n  --trusted-ca-file=/etc/etcd/ca.pem \\\r\n  --peer-trusted-ca-file=/etc/etcd/ca.pem \\\r\n  --initial-advertise-peer-urls https://192.168.19.172:2380 \\\r\n  --listen-peer-urls https://192.168.19.172:2380 \\\r\n  --listen-client-urls https://192.168.19.172:2379,http://127.0.0.1:2379 \\\r\n  --advertise-client-urls https://192.168.19.172:2379 \\\r\n  --initial-cluster-token k8scluster \\\r\n  --initial-cluster K8SNode-Master01=https://192.168.19.171:2380,K8SNode-Master02=https://192.168.19.172:2380,K8SNode-Master03=https://192.168.19.173:2380 \\\r\n  --initial-cluster-state new \\\r\n  --data-dir=/var/lib/etcd\r\nRestart=on-failure\r\nRestartSec=5\r\n\r\n[Install]\r\nWantedBy=multi-user.target\r\n\r\n```\r\n\r\nNode-3 Service file\r\n\r\n```\r\n[Unit]\r\nDescription=etcd\r\nDocumentation=https://github.com/coreos\r\n\r\n[Service]\r\nType=notify\r\nExecStart=/usr/bin/etcd --name K8SNode-Master03 \\\r\n  --cert-file=/etc/etcd/kubernetes.pem \\\r\n  --key-file=/etc/etcd/kubernetes-key.pem \\\r\n  --peer-cert-file=/etc/etcd/kubernetes.pem \\\r\n  --peer-key-file=/etc/etcd/kubernetes-key.pem \\\r\n  --trusted-ca-file=/etc/etcd/ca.pem \\\r\n  --peer-trusted-ca-file=/etc/etcd/ca.pem \\\r\n  --initial-advertise-peer-urls https://192.168.19.173:2380 \\\r\n  --listen-peer-urls https://192.168.19.173:2380 \\\r\n  --listen-client-urls https://192.168.19.173:2379,http://127.0.0.1:2379 \\\r\n  --advertise-client-urls https://192.168.19.173:2379 \\\r\n  --initial-cluster-token k8scluster \\\r\n  --initial-cluster K8SNode-Master01=https://192.168.19.171:2380,K8SNode-Master02=https://192.168.19.172:2380,K8SNode-Master03=https://192.168.19.173:2380 \\\r\n  --initial-cluster-state new \\\r\n  --data-dir=/var/lib/etcd\r\nRestart=on-failure\r\nRestartSec=5\r\n\r\n[Install]\r\nWantedBy=multi-user.target\r\n```\r\n\r\nKubernetes json file entry.\r\n\r\n```\r\n cat kubernetes-csr.json\r\n{\r\n  \"CN\": \"kubernetes\",\r\n  \"hosts\": [\r\n    \"K8SNode-Master01\",\r\n    \"K8SNode-Master02\",\r\n    \"K8SNode-Master03\",\r\n    \"K8SNode-Worker01\",\r\n    \"K8SNode-Worker02\",\r\n    \"K8SNode-Worker03\",\r\n    \"192.168.19.171\",\r\n    \"192.168.19.172\",\r\n    \"192.168.19.173\",\r\n    \"192.168.19.174\",\r\n    \"192.168.19.175\",\r\n    \"192.168.19.176\",\r\n    \"127.0.0.1\"\r\n  ],\r\n  \"key\": {\r\n    \"algo\": \"rsa\",\r\n    \"size\": 2048\r\n  },\r\n  \"names\": [\r\n    {\r\n      \"C\": \"US\",\r\n      \"L\": \"Portland\",\r\n      \"O\": \"Kubernetes\",\r\n      \"OU\": \"Cluster\",\r\n      \"ST\": \"Oregon\"\r\n    }\r\n  ]\r\n}\r\n```\r\n"},"comment":{"url":"https://api.github.com/repos/coreos/etcd/issues/comments/261599342","html_url":"https://github.com/coreos/etcd/issues/6870#issuecomment-261599342","issue_url":"https://api.github.com/repos/coreos/etcd/issues/6870","id":261599342,"user":{"login":"gyuho","id":6799218,"avatar_url":"https://avatars.githubusercontent.com/u/6799218?v=3","gravatar_id":"","url":"https://api.github.com/users/gyuho","html_url":"https://github.com/gyuho","followers_url":"https://api.github.com/users/gyuho/followers","following_url":"https://api.github.com/users/gyuho/following{/other_user}","gists_url":"https://api.github.com/users/gyuho/gists{/gist_id}","starred_url":"https://api.github.com/users/gyuho/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gyuho/subscriptions","organizations_url":"https://api.github.com/users/gyuho/orgs","repos_url":"https://api.github.com/users/gyuho/repos","events_url":"https://api.github.com/users/gyuho/events{/privacy}","received_events_url":"https://api.github.com/users/gyuho/received_events","type":"User","site_admin":false},"created_at":"2016-11-18T18:04:42Z","updated_at":"2016-11-18T18:04:42Z","body":"Try specifying `--key-file` and `--cert-file` in `etcdctl` command?"}}
{"actor":{"display_login":"gyuho"}}