{"action":"created","issue":{"url":"https://api.github.com/repos/projectcalico/calico/issues/978","repository_url":"https://api.github.com/repos/projectcalico/calico","labels_url":"https://api.github.com/repos/projectcalico/calico/issues/978/labels{/name}","comments_url":"https://api.github.com/repos/projectcalico/calico/issues/978/comments","events_url":"https://api.github.com/repos/projectcalico/calico/issues/978/events","html_url":"https://github.com/projectcalico/calico/issues/978","id":132634492,"number":978,"title":"Calico for containers(Kubernetes) not freeing endpoints if minion rebooted","user":{"login":"Nagodar","id":11867259,"avatar_url":"https://avatars.githubusercontent.com/u/11867259?v=3","gravatar_id":"","url":"https://api.github.com/users/Nagodar","html_url":"https://github.com/Nagodar","followers_url":"https://api.github.com/users/Nagodar/followers","following_url":"https://api.github.com/users/Nagodar/following{/other_user}","gists_url":"https://api.github.com/users/Nagodar/gists{/gist_id}","starred_url":"https://api.github.com/users/Nagodar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Nagodar/subscriptions","organizations_url":"https://api.github.com/users/Nagodar/orgs","repos_url":"https://api.github.com/users/Nagodar/repos","events_url":"https://api.github.com/users/Nagodar/events{/privacy}","received_events_url":"https://api.github.com/users/Nagodar/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-02-10T08:25:28Z","updated_at":"2016-02-10T17:10:55Z","closed_at":null,"body":"Hi,\r\n\r\n   I have found that if you reboot kubernetes minions (or restart docker service for example) the endpoints are not freed up.\r\n```\r\nroot@kubcalico01:/var/log# calicoctl endpoint show\r\n+-------------+-----------------+---------------------+---------------------+\r\n|   Hostname  | Orchestrator ID | Number of Workloads | Number of Endpoints |\r\n+-------------+-----------------+---------------------+---------------------+\r\n| kubcalico02 |       cni       |          76         |          76         |\r\n| kubcalico03 |       cni       |          25         |          25         |\r\n+-------------+-----------------+---------------------+---------------------+\r\nroot@kubcalico01:/var/log# kubectl get pods --all-namespaces\r\nNAMESPACE     NAME                                READY     STATUS    RESTARTS   AGE\r\ndefault       kube-proxy-192.168.1.88             1/1       Running   2          17h\r\ndefault       kube-proxy-192.168.1.89             1/1       Running   2          17h\r\nkube-system   elasticsearch-logging-v1-gj3et      1/1       Running   0          15h\r\nkube-system   elasticsearch-logging-v1-md56b      1/1       Running   0          15h\r\nkube-system   fluentd-192.168.1.88                1/1       Running   1          15h\r\nkube-system   fluentd-192.168.1.89                1/1       Running   1          15h\r\nkube-system   kibana-logging-v1-00x2g             1/1       Running   0          15h\r\nkube-system   kube-dns-v11-h6a05                  4/4       Running   1          16h\r\nkube-system   kube-dns-v11-rtt4u                  4/4       Running   0          16h\r\nkube-system   kube-registry-v0-8pa7e              1/1       Running   0          16h\r\nkube-system   kube-ui-v5-sz0d0                    1/1       Running   0          16h\r\nkube-system   kubernetes-dashboard-v0.1.0-bvgu9   1/1       Running   0          16h\r\n```\r\nUsing cni, kubernetes 1.1.7, calico for containers 1.0.0, ubuntu 15.10"},"comment":{"url":"https://api.github.com/repos/projectcalico/calico/issues/comments/182486088","html_url":"https://github.com/projectcalico/calico/issues/978#issuecomment-182486088","issue_url":"https://api.github.com/repos/projectcalico/calico/issues/978","id":182486088,"user":{"login":"caseydavenport","id":1944093,"avatar_url":"https://avatars.githubusercontent.com/u/1944093?v=3","gravatar_id":"","url":"https://api.github.com/users/caseydavenport","html_url":"https://github.com/caseydavenport","followers_url":"https://api.github.com/users/caseydavenport/followers","following_url":"https://api.github.com/users/caseydavenport/following{/other_user}","gists_url":"https://api.github.com/users/caseydavenport/gists{/gist_id}","starred_url":"https://api.github.com/users/caseydavenport/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/caseydavenport/subscriptions","organizations_url":"https://api.github.com/users/caseydavenport/orgs","repos_url":"https://api.github.com/users/caseydavenport/repos","events_url":"https://api.github.com/users/caseydavenport/events{/privacy}","received_events_url":"https://api.github.com/users/caseydavenport/received_events","type":"User","site_admin":false},"created_at":"2016-02-10T17:10:55Z","updated_at":"2016-02-10T17:10:55Z","body":"Hi @Nagodar - I believe you're hitting this upstream Kubernetes issue: https://github.com/kubernetes/kubernetes/issues/14940\r\n\r\nEssentially, the Kubelet isn't telling the network plugin that the old pods have been destroyed, and thus Calico doesn't know to delete those endpoints.\r\n\r\nThis is something that we'd like to fix and contribute back to the Kubernetes project."}}