{"action":"created","issue":{"url":"https://api.github.com/repos/sapcc/kubernikus/issues/284","repository_url":"https://api.github.com/repos/sapcc/kubernikus","labels_url":"https://api.github.com/repos/sapcc/kubernikus/issues/284/labels{/name}","comments_url":"https://api.github.com/repos/sapcc/kubernikus/issues/284/comments","events_url":"https://api.github.com/repos/sapcc/kubernikus/issues/284/events","html_url":"https://github.com/sapcc/kubernikus/issues/284","id":320788271,"number":284,"title":"Monitor kube-proxy health","user":{"login":"databus23","id":439223,"avatar_url":"https://avatars1.githubusercontent.com/u/439223?v=4","gravatar_id":"","url":"https://api.github.com/users/databus23","html_url":"https://github.com/databus23","followers_url":"https://api.github.com/users/databus23/followers","following_url":"https://api.github.com/users/databus23/following{/other_user}","gists_url":"https://api.github.com/users/databus23/gists{/gist_id}","starred_url":"https://api.github.com/users/databus23/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/databus23/subscriptions","organizations_url":"https://api.github.com/users/databus23/orgs","repos_url":"https://api.github.com/users/databus23/repos","events_url":"https://api.github.com/users/databus23/events{/privacy}","received_events_url":"https://api.github.com/users/databus23/received_events","type":"User","site_admin":false},"labels":[{"id":590486182,"url":"https://api.github.com/repos/sapcc/kubernikus/labels/bug","name":"bug","color":"ee0701","default":true}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2018-05-07T12:37:15Z","updated_at":"2018-05-08T06:40:02Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"I fixed a cluster where all kube-proxies had to be restarted. There was no log indicating an error. So my assumption is that they got stuck on a bogus network connection.\r\nWe should try to monitor the state of kube-proxy on nodes.\r\n\r\n`kube-proxy` has a healthcheck url `http://localhost:10249/healthz`. Unfortunately I didn't check if that was reporting an error in the concrete case before restarting the processes."},"comment":{"url":"https://api.github.com/repos/sapcc/kubernikus/issues/comments/387300576","html_url":"https://github.com/sapcc/kubernikus/issues/284#issuecomment-387300576","issue_url":"https://api.github.com/repos/sapcc/kubernikus/issues/284","id":387300576,"user":{"login":"databus23","id":439223,"avatar_url":"https://avatars1.githubusercontent.com/u/439223?v=4","gravatar_id":"","url":"https://api.github.com/users/databus23","html_url":"https://github.com/databus23","followers_url":"https://api.github.com/users/databus23/followers","following_url":"https://api.github.com/users/databus23/following{/other_user}","gists_url":"https://api.github.com/users/databus23/gists{/gist_id}","starred_url":"https://api.github.com/users/databus23/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/databus23/subscriptions","organizations_url":"https://api.github.com/users/databus23/orgs","repos_url":"https://api.github.com/users/databus23/repos","events_url":"https://api.github.com/users/databus23/events{/privacy}","received_events_url":"https://api.github.com/users/databus23/received_events","type":"User","site_admin":false},"created_at":"2018-05-08T06:40:02Z","updated_at":"2018-05-08T06:40:02Z","author_association":"CONTRIBUTOR","body":"@auhlig We need to prioritise this. Looking at some of our control plane clusters we have quite a few proxies in bad shape. First thing would be to figure out if we can ingest the kube-proxy metrics and see something useful if it fails"}}
{"actor":{"display_login":"databus23"}}