{"action":"created","issue":{"url":"https://api.github.com/repos/coreos/etcd/issues/6000","repository_url":"https://api.github.com/repos/coreos/etcd","labels_url":"https://api.github.com/repos/coreos/etcd/issues/6000/labels{/name}","comments_url":"https://api.github.com/repos/coreos/etcd/issues/6000/comments","events_url":"https://api.github.com/repos/coreos/etcd/issues/6000/events","html_url":"https://github.com/coreos/etcd/issues/6000","id":166480474,"number":6000,"title":"All etcd2 server exits with static mode.","user":{"login":"Yancey1989","id":1426912,"avatar_url":"https://avatars.githubusercontent.com/u/1426912?v=3","gravatar_id":"","url":"https://api.github.com/users/Yancey1989","html_url":"https://github.com/Yancey1989","followers_url":"https://api.github.com/users/Yancey1989/followers","following_url":"https://api.github.com/users/Yancey1989/following{/other_user}","gists_url":"https://api.github.com/users/Yancey1989/gists{/gist_id}","starred_url":"https://api.github.com/users/Yancey1989/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Yancey1989/subscriptions","organizations_url":"https://api.github.com/users/Yancey1989/orgs","repos_url":"https://api.github.com/users/Yancey1989/repos","events_url":"https://api.github.com/users/Yancey1989/events{/privacy}","received_events_url":"https://api.github.com/users/Yancey1989/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2016-07-20T02:25:07Z","updated_at":"2016-07-20T06:26:46Z","closed_at":null,"body":"#### Environment\r\n```\r\nbjlg-49p45-k8s-01 static-etcd2 # cat /etc/os-release\r\nNAME=CoreOS\r\nID=coreos\r\nVERSION=1010.6.0\r\nVERSION_ID=1010.6.0\r\nBUILD_ID=2016-06-28-0910\r\nPRETTY_NAME=\"CoreOS 1010.6.0 (MoreOS)\"\r\nANSI_COLOR=\"1;32\"\r\nHOME_URL=\"https://coreos.com/\"\r\nBUG_REPORT_URL=\"https://github.com/coreos/bugs/issues\"\r\nbjlg-49p45-k8s-01 static-etcd2 # /usr/bin/etcd2 -version\r\netcd Version: 2.3.1\r\nGit SHA: 2b67f52\r\nGo Version: go1.5.3\r\nGo OS/Arch: linux/amd64\r\n```\r\n#### Deploy bash script, 3 etcd memory with named infra0, infra1 and infra2\r\n\r\n```\r\nexport public_ipv4_infra0=192.168.49.45\r\nexport public_ipv4_infra1=192.168.49.46\r\nexport public_ipv4_infra2=192.168.49.47\r\nexport name=infra0\r\nexport NITIAL_CLUSTER=\"infra0=http://$public_ipv4_infra0:2380,infra1=http://$public_ipv4_infra1:2380,infra2=http://$public_ipv4_infra2:2380\"\r\nexport ETCD_INITIAL_CLUSTER_STATE=new\r\n/usr/bin/etcd2 --name $name --initial-advertise-peer-urls http://$public_ipv4_infra0:2380 \\\r\n  --listen-peer-urls http://$public_ipv4_infra0:2380 \\\r\n  --listen-client-urls http://$public_ipv4_infra0:2379,http://127.0.0.1:2379 \\\r\n  --advertise-client-urls http://$public_ipv4_infra0:2379 \\\r\n  --initial-cluster-token etcd-cluster-1 \\\r\n  --initial-cluster infra0=http://$public_ipv4_infra0:2380,infra1=http://$public_ipv4_infra1:2380,infra2=http://$public_ipv4_infra2:2380 \\\r\n  --initial-cluster-state new \\\r\n  --data-dir /var/lib/etcd2/$name \\\r\n```\r\n\r\n####  All process will be existed after a period of time. Here is the logs\r\n\r\ninfra0 log\r\n```\r\n2016-07-19 14:29:24.289951 E | etcdhttp: got unexpected response error (etcdserver: request timed out)\r\n2016-07-19 14:29:24.859851 I | raft: f3aefe338854c13a is starting a new election at term 3419\r\n2016-07-19 14:29:24.859887 I | raft: f3aefe338854c13a became candidate at term 3420\r\n2016-07-19 14:29:24.859910 I | raft: f3aefe338854c13a received vote from f3aefe338854c13a at term 3420\r\n2016-07-19 14:29:24.859923 I | raft: f3aefe338854c13a [logterm: 3243, index: 262126] sent vote request to e344a1396881242e at term 3420\r\n2016-07-19 14:29:24.859935 I | raft: f3aefe338854c13a [logterm: 3243, index: 262126] sent vote request to eff15145dca03741 at term 3420\r\n2016-07-19 14:29:26.000041 E | etcdhttp: got unexpected response error (etcdserver: request timed out) [merged 3 repeated lines in 1.71s]\r\n2016-07-19 14:29:26.159758 I | raft: f3aefe338854c13a is starting a new election at term 3420\r\n2016-07-19 14:29:26.159803 I | raft: f3aefe338854c13a became candidate at term 3421\r\n2016-07-19 14:29:26.159814 I | raft: f3aefe338854c13a received vote from f3aefe338854c13a at term 3421\r\n2016-07-19 14:29:26.159828 I | raft: f3aefe338854c13a [logterm: 3243, index: 262126] sent vote request to e344a1396881242e at term 3421\r\n2016-07-19 14:29:26.159841 I | raft: f3aefe338854c13a [logterm: 3243, index: 262126] sent vote request to eff15145dca03741 at term 3421\r\n2016-07-19 14:29:27.559756 I | raft: f3aefe338854c13a is starting a new election at term 3421\r\n2016-07-19 14:29:27.559806 I | raft: f3aefe338854c13a became candidate at term 3422\r\n2016-07-19 14:29:27.559818 I | raft: f3aefe338854c13a received vote from f3aefe338854c13a at term 3422\r\n2016-07-19 14:29:27.559831 I | raft: f3aefe338854c13a [logterm: 3243, index: 262126] sent vote request to e344a1396881242e at term 3422\r\n2016-07-19 14:29:27.559844 I | raft: f3aefe338854c13a [logterm: 3243, index: 262126] sent vote request to eff15145dca03741 at term 3422\r\n2016-07-19 14:29:29.059778 I | raft: f3aefe338854c13a is starting a new election at term 3422\r\n2016-07-19 14:29:29.059813 I | raft: f3aefe338854c13a became candidate at term 3423\r\n2016-07-19 14:29:29.059824 I | raft: f3aefe338854c13a received vote from f3aefe338854c13a at term 3423\r\n2016-07-19 14:29:29.059838 I | raft: f3aefe338854c13a [logterm: 3243, index: 262126] sent vote request to e344a1396881242e at term 3423\r\n2016-07-19 14:29:29.059850 I | raft: f3aefe338854c13a [logterm: 3243, index: 262126] sent vote request to eff15145dca03741 at term 3423\r\n```\r\ninfra1 log\r\n\r\n```\r\n2016-07-19 10:41:53.820550 I | fileutil: purged file /var/lib/etcd2/infra1/member/snap/0000000000000caa-0000000000029f36.snap successfully\r\n2016-07-19 11:30:29.568414 I | raft: e344a1396881242e [term: 3242] received a MsgVote message with higher term from eff15145dca03741 [term: 3243]\r\n2016-07-19 11:30:29.568457 I | raft: e344a1396881242e became follower at term 3243\r\n2016-07-19 11:30:29.568474 I | raft: e344a1396881242e [logterm: 3242, index: 230649, vote: 0] voted for eff15145dca03741 [logterm: 3242, index: 230649] at term 3243\r\n2016-07-19 11:30:29.568485 I | raft: raft.node: e344a1396881242e lost leader f3aefe338854c13a at term 3243\r\n2016-07-19 11:30:29.578578 I | raft: raft.node: e344a1396881242e elected leader eff15145dca03741 at term 3243\r\n2016-07-19 11:30:30.119368 I | raft: e344a1396881242e [term: 3243] ignored a MsgApp message with lower term from f3aefe338854c13a [term: 3242]\r\n2016-07-19 11:30:30.119402 I | raft: e344a1396881242e [term: 3243] ignored a MsgApp message with lower term from f3aefe338854c13a [term: 3242]\r\n2016-07-19 11:37:03.513093 I | etcdserver: start to snapshot (applied: 231836, lastsnap: 221835)\r\n2016-07-19 11:37:03.545078 I | etcdserver: saved snapshot at index 231836\r\n2016-07-19 11:37:03.545449 I | etcdserver: compacted raft log at 226836\r\n2016-07-19 11:37:23.839884 I | fileutil: purged file /var/lib/etcd2/infra1/member/snap/0000000000000caa-000000000002c647.snap successfully\r\n2016-07-19 12:32:38.217257 I | etcdserver: start to snapshot (applied: 241837, lastsnap: 231836)\r\n2016-07-19 12:32:38.249246 I | etcdserver: saved snapshot at index 241837\r\n2016-07-19 12:32:38.249525 I | etcdserver: compacted raft log at 236837\r\n2016-07-19 12:32:53.859343 I | fileutil: purged file /var/lib/etcd2/infra1/member/snap/0000000000000caa-000000000002ed58.snap successfully\r\n2016-07-19 13:28:11.867890 I | etcdserver: start to snapshot (applied: 251838, lastsnap: 241837)\r\n2016-07-19 13:28:11.899176 I | etcdserver: saved snapshot at index 251838\r\n2016-07-19 13:28:11.899461 I | etcdserver: compacted raft log at 246838\r\n2016-07-19 13:28:23.876017 I | fileutil: purged file /var/lib/etcd2/infra1/member/snap/0000000000000caa-0000000000031469.snap successfully\r\n2016-07-19 14:23:45.867896 I | etcdserver: start to snapshot (applied: 261839, lastsnap: 251838)\r\n2016-07-19 14:23:45.900204 I | etcdserver: saved snapshot at index 261839\r\n2016-07-19 14:23:45.900482 I | etcdserver: compacted raft log at 256839\r\n2016-07-19 14:23:53.894054 I | fileutil: purged file /var/lib/etcd2/infra1/member/snap/0000000000000caa-0000000000033b7a.snap successfully\r\n```\r\ninfra2 log\r\n```\r\n2016-07-19 14:25:00.608974 W | rafthttp: the connection to peer e344a1396881242e is unhealthy\r\n2016-07-19 14:25:00.842435 W | etcdserver: failed to reach the peerURL(http://192.168.49.46:2380) of member e344a1396881242e (Get http://192.168.49.46:2380/version: dial tcp 192.168.49.46:2380: getsockopt: connection refused)\r\n2016-07-19 14:25:00.842488 W | etcdserver: cannot get the version of member e344a1396881242e (Get http://192.168.49.46:2380/version: dial tcp 192.168.49.46:2380: getsockopt: connection refused)\r\n2016-07-19 14:25:04.843764 W | etcdserver: failed to reach the peerURL(http://192.168.49.46:2380) of member e344a1396881242e (Get http://192.168.49.46:2380/version: dial tcp 192.168.49.46:2380: getsockopt: connection refused)\r\n2016-07-19 14:25:04.843825 W | etcdserver: cannot get the version of member e344a1396881242e (Get http://192.168.49.46:2380/version: dial tcp 192.168.49.46:2380: getsockopt: connection refused)\r\n2016-07-19 14:25:08.845047 W | etcdserver: failed to reach the peerURL(http://192.168.49.46:2380) of member e344a1396881242e (Get http://192.168.49.46:2380/version: dial tcp 192.168.49.46:2380: getsockopt: connection refused)\r\n2016-07-19 14:25:08.845106 W | etcdserver: cannot get the version of member e344a1396881242e (Get http://192.168.49.46:2380/version: dial tcp 192.168.49.46:2380: getsockopt: connection refused)\r\n2016-07-19 14:25:12.846185 W | etcdserver: failed to reach the peerURL(http://192.168.49.46:2380) of member e344a1396881242e (Get http://192.168.49.46:2380/version: dial tcp 192.168.49.46:2380: getsockopt: connection refused)\r\n2016-07-19 14:25:12.846248 W | etcdserver: cannot get the version of member e344a1396881242e (Get http://192.168.49.46:2380/version: dial tcp 192.168.49.46:2380: getsockopt: connection refused)\r\n2016-07-19 14:25:16.847425 W | etcdserver: failed to reach the peerURL(http://192.168.49.46:2380) of member e344a1396881242e (Get http://192.168.49.46:2380/version: dial tcp 192.168.49.46:2380: getsockopt: connection refused)\r\n2016-07-19 14:25:16.847487 W | etcdserver: cannot get the version of member e344a1396881242e (Get http://192.168.49.46:2380/version: dial tcp 192.168.49.46:2380: getsockopt: connection refused)\r\n2016-07-19 14:25:20.848772 W | etcdserver: failed to reach the peerURL(http://192.168.49.46:2380) of member e344a1396881242e (Get http://192.168.49.46:2380/version: dial tcp 192.168.49.46:2380: getsockopt: connection refused)\r\n2016-07-19 14:25:20.848834 W | etcdserver: cannot get the version of member e344a1396881242e (Get http://192.168.49.46:2380/version: dial tcp 192.168.49.46:2380: getsockopt: connection refused)\r\n2016-07-19 14:25:23.777587 N | osutil: received terminated signal, shutting down...\r\n```"},"comment":{"url":"https://api.github.com/repos/coreos/etcd/issues/comments/233851301","html_url":"https://github.com/coreos/etcd/issues/6000#issuecomment-233851301","issue_url":"https://api.github.com/repos/coreos/etcd/issues/6000","id":233851301,"user":{"login":"Yancey1989","id":1426912,"avatar_url":"https://avatars.githubusercontent.com/u/1426912?v=3","gravatar_id":"","url":"https://api.github.com/users/Yancey1989","html_url":"https://github.com/Yancey1989","followers_url":"https://api.github.com/users/Yancey1989/followers","following_url":"https://api.github.com/users/Yancey1989/following{/other_user}","gists_url":"https://api.github.com/users/Yancey1989/gists{/gist_id}","starred_url":"https://api.github.com/users/Yancey1989/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Yancey1989/subscriptions","organizations_url":"https://api.github.com/users/Yancey1989/orgs","repos_url":"https://api.github.com/users/Yancey1989/repos","events_url":"https://api.github.com/users/Yancey1989/events{/privacy}","received_events_url":"https://api.github.com/users/Yancey1989/received_events","type":"User","site_admin":false},"created_at":"2016-07-20T06:26:46Z","updated_at":"2016-07-20T06:26:46Z","body":"@heyitsanthony thanks for your reply.\r\nthe last logs with **infra1**  stop on\r\n```\r\n2016-07-19 14:23:53.894054 I | fileutil: purged file /var/lib/etcd2/infra1/member/snap/0000000000000caa-0000000000033b7a.snap successfully\r\n```\r\n\r\ninfra0:\r\n```\r\n2016-07-19 14:13:48.628550 I | fileutil: purged file /var/lib/etcd2/infra0/member/snap/0000000000000caa-0000000000033465.snap successfully\r\n2016-07-19 14:24:18.545112 E | rafthttp: failed to read e344a1396881242e on stream MsgApp v2 (unexpected EOF)\r\n2016-07-19 14:24:18.545144 I | rafthttp: the connection with e344a1396881242e became inactive\r\n2016-07-19 14:24:48.536554 W | rafthttp: the connection to peer e344a1396881242e is unhealthy\r\n2016-07-19 14:25:18.536704 W | rafthttp: the connection to peer e344a1396881242e is unhealthy\r\n2016-07-19 14:25:23.783116 E | rafthttp: failed to read eff15145dca03741 on stream Message (unexpected EOF)\r\n2016-07-19 14:25:23.783154 I | rafthttp: the connection with eff15145dca03741 became inactive\r\n2016-07-19 14:25:24.859856 I | raft: f3aefe338854c13a is starting a new election at term 3243\r\n2016-07-19 14:25:24.859900 I | raft: f3aefe338854c13a became candidate at term 3244\r\n2016-07-19 14:25:24.859912 I | raft: f3aefe338854c13a received vote from f3aefe338854c13a at term 3244\r\n2016-07-19 14:25:24.859932 I | raft: f3aefe338854c13a [logterm: 3243, index: 262126] sent vote request to e344a1396881242e at term 3244\r\n2016-07-19 14:25:24.859945 I | raft: f3aefe338854c13a [logterm: 3243, index: 262126] sent vote request to eff15145dca03741 at term 3244\r\n2016-07-19 14:25:24.859963 I | raft: raft.node: f3aefe338854c13a lost leader eff15145dca03741 at term 3244\r\n2016-07-19 14:25:25.959918 I | raft: f3aefe338854c13a is starting a new election at term 3244\r\n2016-07-19 14:25:25.959978 I | raft: f3aefe338854c13a became candidate at term 3245\r\n2016-07-19 14:25:25.959991 I | raft: f3aefe338854c13a received vote from f3aefe338854c13a at term 3245\r\n2016-07-19 14:25:25.960006 I | raft: f3aefe338854c13a [logterm: 3243, index: 262126] sent vote request to e344a1396881242e at term 3245\r\n2016-07-19 14:25:25.960018 I | raft: f3aefe338854c13a [logterm: 3243, index: 262126] sent vote request to eff15145dca03741 at term 3245\r\n2016-07-19 14:25:27.359785 I | raft: f3aefe338854c13a is starting a new election at term 3245\r\n2016-07-19 14:25:27.359841 I | raft: f3aefe338854c13a became candidate at term 3246\r\n2016-07-19 14:25:27.359854 I | raft: f3aefe338854c13a received vote from f3aefe338854c13a at term 3246\r\n2016-07-19 14:25:27.359867 I | raft: f3aefe338854c13a [logterm: 3243, index: 262126] sent vote request to eff15145dca03741 at term 3246\r\n2016-07-19 14:25:27.359879 I | raft: f3aefe338854c13a [logterm: 3243, index: 262126] sent vote request to e344a1396881242e at term 3246\r\n2016-07-19 14:25:28.859792 I | raft: f3aefe338854c13a is starting a new election at term 3246\r\n```"}}
{"actor":{"display_login":"Yancey1989"}}