{"action":"created","issue":{"url":"https://api.github.com/repos/coreos/etcd/issues/4517","repository_url":"https://api.github.com/repos/coreos/etcd","labels_url":"https://api.github.com/repos/coreos/etcd/issues/4517/labels{/name}","comments_url":"https://api.github.com/repos/coreos/etcd/issues/4517/comments","events_url":"https://api.github.com/repos/coreos/etcd/issues/4517/events","html_url":"https://github.com/coreos/etcd/issues/4517","id":133390926,"number":4517,"title":"functional-test: tester client error","user":{"login":"gyuho","id":6799218,"avatar_url":"https://avatars.githubusercontent.com/u/6799218?v=3","gravatar_id":"","url":"https://api.github.com/users/gyuho","html_url":"https://github.com/gyuho","followers_url":"https://api.github.com/users/gyuho/followers","following_url":"https://api.github.com/users/gyuho/following{/other_user}","gists_url":"https://api.github.com/users/gyuho/gists{/gist_id}","starred_url":"https://api.github.com/users/gyuho/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gyuho/subscriptions","organizations_url":"https://api.github.com/users/gyuho/orgs","repos_url":"https://api.github.com/users/gyuho/repos","events_url":"https://api.github.com/users/gyuho/events{/privacy}","received_events_url":"https://api.github.com/users/gyuho/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/coreos/etcd/labels/area/testing","name":"area/testing","color":"bfe5bf"}],"state":"open","locked":false,"assignee":{"login":"gyuho","id":6799218,"avatar_url":"https://avatars.githubusercontent.com/u/6799218?v=3","gravatar_id":"","url":"https://api.github.com/users/gyuho","html_url":"https://github.com/gyuho","followers_url":"https://api.github.com/users/gyuho/followers","following_url":"https://api.github.com/users/gyuho/following{/other_user}","gists_url":"https://api.github.com/users/gyuho/gists{/gist_id}","starred_url":"https://api.github.com/users/gyuho/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gyuho/subscriptions","organizations_url":"https://api.github.com/users/gyuho/orgs","repos_url":"https://api.github.com/users/gyuho/repos","events_url":"https://api.github.com/users/gyuho/events{/privacy}","received_events_url":"https://api.github.com/users/gyuho/received_events","type":"User","site_admin":false},"milestone":null,"comments":25,"created_at":"2016-02-13T02:55:51Z","updated_at":"2016-02-20T07:42:26Z","closed_at":null,"body":"There was a bad connection in client side. This is not an etcd error. It might be worth having some retrial logic.\r\n\r\n```\r\n2016/02/13 01:12:51 etcd-tester: [round#83 case#2] recovery error: rpc error: code = 4 desc = \"context deadline exceeded\"\r\n2016/02/13 01:12:51 etcd-tester: [round#83 case#2] cleaning up...\r\n2016/02/13 01:12:53 etcd-tester: [round#83 case#3] start failure kill leader member\r\n2016/02/13 01:12:53 etcd-tester: [round#83 case#3] start injecting failure...\r\n2016/02/13 01:12:53 etcd-tester: [round#83 case#3] injected failure\r\n2016/02/13 01:12:53 etcd-tester: [round#83 case#3] start recovering failure...\r\n2016/02/13 01:12:56 etcd-tester: [round#83 case#3] recovered failure\r\n```"},"comment":{"url":"https://api.github.com/repos/coreos/etcd/issues/comments/186537013","html_url":"https://github.com/coreos/etcd/issues/4517#issuecomment-186537013","issue_url":"https://api.github.com/repos/coreos/etcd/issues/4517","id":186537013,"user":{"login":"gyuho","id":6799218,"avatar_url":"https://avatars.githubusercontent.com/u/6799218?v=3","gravatar_id":"","url":"https://api.github.com/users/gyuho","html_url":"https://github.com/gyuho","followers_url":"https://api.github.com/users/gyuho/followers","following_url":"https://api.github.com/users/gyuho/following{/other_user}","gists_url":"https://api.github.com/users/gyuho/gists{/gist_id}","starred_url":"https://api.github.com/users/gyuho/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gyuho/subscriptions","organizations_url":"https://api.github.com/users/gyuho/orgs","repos_url":"https://api.github.com/users/gyuho/repos","events_url":"https://api.github.com/users/gyuho/events{/privacy}","received_events_url":"https://api.github.com/users/gyuho/received_events","type":"User","site_admin":false},"created_at":"2016-02-20T07:42:26Z","updated_at":"2016-02-20T07:42:26Z","body":"@xiang90 I think https://github.com/coreos/etcd/pull/4572 fixed this issue. I redeployed with your patch and port usage stays stable now.\r\n\r\nFor round 1, we start with about 5 ~ 6 gRPC connections.\r\n\r\n```\r\nstresser count: map[10.240.0.2:2378:200 10.240.0.4:2378:200 10.240.0.3:2378:200]\r\ngRPC port count: map[10.240.0.3:2378:5 10.240.0.4:2378:5 10.240.0.2:2378:6]\r\n```\r\n\r\nAnd after 90 rounds, the gRPC connection usage is same, which is expected behavior:\r\n\r\n```\r\nstresser count: map[10.240.0.2:2378:200 10.240.0.4:2378:200 10.240.0.3:2378:200]\r\n gRPC port count: map[10.240.0.2:2378:20 10.240.0.4:2378:3 10.240.0.3:2378:3]\r\n```\r\n\r\n"}}