{"action":"created","issue":{"url":"https://api.github.com/repos/kubernetes/kubernetes/issues/21451","repository_url":"https://api.github.com/repos/kubernetes/kubernetes","labels_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/21451/labels{/name}","comments_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/21451/comments","events_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/21451/events","html_url":"https://github.com/kubernetes/kubernetes/issues/21451","id":134476523,"number":21451,"title":"\"You cannot remove a running container.\" during hack/test-cmd.sh or integration tests","user":{"login":"zmerlynn","id":4942464,"avatar_url":"https://avatars.githubusercontent.com/u/4942464?v=3","gravatar_id":"","url":"https://api.github.com/users/zmerlynn","html_url":"https://github.com/zmerlynn","followers_url":"https://api.github.com/users/zmerlynn/followers","following_url":"https://api.github.com/users/zmerlynn/following{/other_user}","gists_url":"https://api.github.com/users/zmerlynn/gists{/gist_id}","starred_url":"https://api.github.com/users/zmerlynn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zmerlynn/subscriptions","organizations_url":"https://api.github.com/users/zmerlynn/orgs","repos_url":"https://api.github.com/users/zmerlynn/repos","events_url":"https://api.github.com/users/zmerlynn/events{/privacy}","received_events_url":"https://api.github.com/users/zmerlynn/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/kubernetes/kubernetes/labels/area/test","name":"area/test","color":"fad8c7"},{"url":"https://api.github.com/repos/kubernetes/kubernetes/labels/kind/flake","name":"kind/flake","color":"f7c6c7"},{"url":"https://api.github.com/repos/kubernetes/kubernetes/labels/priority/P0","name":"priority/P0","color":"e11d21"},{"url":"https://api.github.com/repos/kubernetes/kubernetes/labels/team/node","name":"team/node","color":"d2b48c"},{"url":"https://api.github.com/repos/kubernetes/kubernetes/labels/team/test-infra","name":"team/test-infra","color":"d2b48c"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":18,"created_at":"2016-02-18T04:09:00Z","updated_at":"2016-02-20T03:15:41Z","closed_at":null,"body":"From [build-log.txt](https://00e9e64bac6b200a32447dc4f63b6030019cc0bc3a44f16a27-apidata.googleusercontent.com/download/storage/v1_internal/b/kubernetes-jenkins/o/pr-logs%2Fpull%2F21439%2Fkubernetes-pull-test-unit-integration%2F14897%2Fbuild-log.txt?qk=AD5uMEvhvtd0nU5LWkcTBK9YLg-hvrEQTWAt9cK36vzC3Zy2HiqUXBVZSnxKHiCp67X0z0Xd-JeEOgnBtccHilLhlzARzAhmuSSrBVr6Sssz4EvF10NRGnV6OpUdMT4VHvSiR4wt_U8vT66ZBARu7PucUuFuQQVnROKuqqXqcmLLyfR5VMD1uPfcE-oWhIGvuiDqnwACkUj4zikLoQa1E9D5Opnz6XKovcPH_KO17jL6y6nioOIFphUdGAQ9Ogs1Ikncr1uA4kAarWbGkij4edu1FwKiQGuY2LO1idJwDWIRNkZwf5LKacsDEw1QnDAi_0TqR94ckcR8ZzQ4SxqxxVIM5TKqkkskyD620HMvL2zgX-bjNnMuLEEsWXEYvzluupu9qTemAXRHxg9i7WfUkqk4gI3YWbxp-2BwCfoimtxVzeaACfVMFOQrihSNKc9dacf7CQZeDnUGpLumteoCehJehE09pX2pM8P4dPnuiUPNXgYDZLBcvyc8ftSMJJ1q_fZ7R1TTgVXdVkt4O3CjFBVjeBbU-ngH6DLFmns-x11gw_vaAGyLSmLzp7oHQE0cpBM2-6yJglezpJTcQ5MTVUVDNfziVud-PZ8wK59zJJRPkDoxmq8FQmmoci-fuZWUEoQM1atW0PvjRJZOFNY2Bwo91TWGA7lIFdJxCiVD0ehMSFo9YV4JUuRuQpE0mHeM6sS-DYPc3_Pf2Cm9ll029iiCq6NTTbmYZ4lnKcFZE3VnVvL3X8NaG6kHz3yQ1TlGon0hkSUDg0ZEuTNpOhWQUHxGAuV4rXMKDSF3xj1dt3_bulKbx3v1op9ILbQYCda3XrPypKpJGFiwmAPo1FwHi0tjkFlLJh7IzLRRR2xplm_Mpe3Y5lT2C7ikJHP32aAY360PhujaDB18oF8ORHre3aG_ljlm-CBNod4BqX7n4loZacseJa4h_GE) in #21439 (before I retest) (Tried to find a flake dupe but didn't see an obvious one):\r\n\r\n```\r\nI0217 18:27:49.485036   12269 manager.go:1620] Found pod infra container for \"phantom.baz_default(3324bd82-d5e7-11e5-bf2a-0242ac110003)\"\r\nI0217 18:27:49.485120   12269 manager.go:1633] Pod infra container looks good, keep it \"phantom.baz_default(3324bd82-d5e7-11e5-bf2a-0242ac110003)\"\r\nI0217 18:27:49.485169   12269 manager.go:1657] pod \"phantom.baz_default(3324bd82-d5e7-11e5-bf2a-0242ac110003)\" container \"c1\" exists as /k8s_c1.cd1b02d5_phantom.baz_default_3324bd82-d5e7-11e5-bf2a-0242ac110003_46090a9a\r\nI0217 18:27:49.485201   12269 manager.go:1725] Got container changes for pod \"phantom.baz_default(3324bd82-d5e7-11e5-bf2a-0242ac110003)\": {StartInfraContainer:false InfraChanged:false InfraContainerId:/k8s_POD.6d140183_phantom.baz_default_3324bd82-d5e7-11e5-bf2a-0242ac110003_e7f1f119 ContainersToStart:map[] ContainersToKeep:map[/k8s_POD.6d140183_phantom.baz_default_3324bd82-d5e7-11e5-bf2a-0242ac110003_e7f1f119:-1 /k8s_c1.cd1b02d5_phantom.baz_default_3324bd82-d5e7-11e5-bf2a-0242ac110003_46090a9a:0]}\r\nI0217 18:27:49.692180   12269 integration.go:958] Scheduler doesn't make phantom pods: test passed.\r\nI0217 18:27:49.692211   12269 integration.go:1068] \r\n\r\nLogging high latency metrics from the 10250 kubelet\r\nI0217 18:27:49.716299   12269 server.go:1083] GET /metrics: (23.237036ms) 200 [[Go-http-client/1.1] 127.0.0.1:56949]\r\n\u0002������iI0217 18:27:49.725600   12269 integration.go:1070] \r\n\r\nLogging high latency metrics from the 10251 kubelet\r\nFeb 17 18:27:49.725: INFO: \r\nLatency metrics for node localhost:10250\r\nFeb 17 18:27:49.725: INFO: {Operation: Method:pod_start_latency_microseconds Quantile:0.99 Latency:6.002896s}\r\nFeb 17 18:27:49.725: INFO: {Operation: Method:pod_start_latency_microseconds Quantile:0.9 Latency:5.002607s}\r\nFeb 17 18:27:49.725: INFO: {Operation: Method:pod_worker_start_latency_microseconds Quantile:0.99 Latency:5.002121s}\r\nFeb 17 18:27:49.725: INFO: {Operation: Method:pod_worker_start_latency_microseconds Quantile:0.9 Latency:5.002121s}\r\nFeb 17 18:27:49.725: INFO: {Operation:sync Method:pod_worker_latency_microseconds Quantile:0.99 Latency:1.001256s}\r\nError deleting container: Error response from daemon: Conflict, You cannot remove a running container. Stop the container before attempting removal or use -f\r\nUnrecognized input header\r\nBuild step 'Execute shell' marked build as failure\r\nRecording test results\r\n```"},"comment":{"url":"https://api.github.com/repos/kubernetes/kubernetes/issues/comments/186496812","html_url":"https://github.com/kubernetes/kubernetes/issues/21451#issuecomment-186496812","issue_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/21451","id":186496812,"user":{"login":"bprashanth","id":10927820,"avatar_url":"https://avatars.githubusercontent.com/u/10927820?v=3","gravatar_id":"","url":"https://api.github.com/users/bprashanth","html_url":"https://github.com/bprashanth","followers_url":"https://api.github.com/users/bprashanth/followers","following_url":"https://api.github.com/users/bprashanth/following{/other_user}","gists_url":"https://api.github.com/users/bprashanth/gists{/gist_id}","starred_url":"https://api.github.com/users/bprashanth/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bprashanth/subscriptions","organizations_url":"https://api.github.com/users/bprashanth/orgs","repos_url":"https://api.github.com/users/bprashanth/repos","events_url":"https://api.github.com/users/bprashanth/events{/privacy}","received_events_url":"https://api.github.com/users/bprashanth/received_events","type":"User","site_admin":false},"created_at":"2016-02-20T03:15:41Z","updated_at":"2016-02-20T03:15:41Z","body":"> Is it possible that the first kubelet is taking a while to exit for some reason, and then ends up trying to delete the cAdvisor container of the second kubelet? \r\n\r\nThe second kubelet should compete for ports with the first (releasing the port on sigterm should be one of the last things it does).\r\n\r\nIs test-cmd running in a container? cadvisor is just printing out its cgroup (/proc/self/cgroup), I believe they (kubelet and cadvisor) run in the same process, so we could still be back at: something is deleting the container we're running the test in. "}}