{"action":"created","issue":{"url":"https://api.github.com/repos/uswitch/kiam/issues/94","repository_url":"https://api.github.com/repos/uswitch/kiam","labels_url":"https://api.github.com/repos/uswitch/kiam/issues/94/labels{/name}","comments_url":"https://api.github.com/repos/uswitch/kiam/issues/94/comments","events_url":"https://api.github.com/repos/uswitch/kiam/issues/94/events","html_url":"https://github.com/uswitch/kiam/issues/94","id":333464196,"node_id":"MDU6SXNzdWUzMzM0NjQxOTY=","number":94,"title":"Kiam server readiness/liveness probe issues (v2.8)","user":{"login":"mattreyuk","id":926405,"node_id":"MDQ6VXNlcjkyNjQwNQ==","avatar_url":"https://avatars3.githubusercontent.com/u/926405?v=4","gravatar_id":"","url":"https://api.github.com/users/mattreyuk","html_url":"https://github.com/mattreyuk","followers_url":"https://api.github.com/users/mattreyuk/followers","following_url":"https://api.github.com/users/mattreyuk/following{/other_user}","gists_url":"https://api.github.com/users/mattreyuk/gists{/gist_id}","starred_url":"https://api.github.com/users/mattreyuk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mattreyuk/subscriptions","organizations_url":"https://api.github.com/users/mattreyuk/orgs","repos_url":"https://api.github.com/users/mattreyuk/repos","events_url":"https://api.github.com/users/mattreyuk/events{/privacy}","received_events_url":"https://api.github.com/users/mattreyuk/received_events","type":"User","site_admin":false},"labels":[{"id":616797778,"node_id":"MDU6TGFiZWw2MTY3OTc3Nzg=","url":"https://api.github.com/repos/uswitch/kiam/labels/bug","name":"bug","color":"ee0701","default":true}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/uswitch/kiam/milestones/1","html_url":"https://github.com/uswitch/kiam/milestone/1","labels_url":"https://api.github.com/repos/uswitch/kiam/milestones/1/labels","id":3404377,"node_id":"MDk6TWlsZXN0b25lMzQwNDM3Nw==","number":1,"title":"3.0","description":"Spurred by the recent activity I'd love to get a 3.0 release finished that made some more significant improvements/changes.","creator":{"login":"pingles","id":1150,"node_id":"MDQ6VXNlcjExNTA=","avatar_url":"https://avatars0.githubusercontent.com/u/1150?v=4","gravatar_id":"","url":"https://api.github.com/users/pingles","html_url":"https://github.com/pingles","followers_url":"https://api.github.com/users/pingles/followers","following_url":"https://api.github.com/users/pingles/following{/other_user}","gists_url":"https://api.github.com/users/pingles/gists{/gist_id}","starred_url":"https://api.github.com/users/pingles/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pingles/subscriptions","organizations_url":"https://api.github.com/users/pingles/orgs","repos_url":"https://api.github.com/users/pingles/repos","events_url":"https://api.github.com/users/pingles/events{/privacy}","received_events_url":"https://api.github.com/users/pingles/received_events","type":"User","site_admin":false},"open_issues":2,"closed_issues":12,"state":"open","created_at":"2018-06-06T08:28:22Z","updated_at":"2018-12-06T10:39:54Z","due_on":"2018-08-10T07:00:00Z","closed_at":null},"comments":45,"created_at":"2018-06-18T23:01:51Z","updated_at":"2018-12-06T10:39:54Z","closed_at":"2018-12-06T10:39:54Z","author_association":"NONE","body":"I have Kiam setup on a 3 master cluster - the server runs on two of the three masters with some readiness probes failing but the 3rd node is in a crash loop backoff with the liveness probe failing. The error message is the same for both:\r\n```\r\nprobe failed: time=\"2018-06-18T22:10:41Z\" level=fatal msg=\"error creating server gateway: error waiting for address being available in the balancer: context deadline exceeded\"\r\n```\r\n Looking in the server logs for the failing pod, I just see it startup, lots of `found role`, `requesting credentials` and then finally `stopped`.\r\nI'm using Image: `quay.io/uswitch/kiam:v2.8` and the standard probe definitions from `/deploy/server.yaml` (this is a `kops` 1.9 cluster so I changed the cert path to `/etc/ssl/certs` and added master tolerations to get it to work). "},"comment":{"url":"https://api.github.com/repos/uswitch/kiam/issues/comments/444827955","html_url":"https://github.com/uswitch/kiam/issues/94#issuecomment-444827955","issue_url":"https://api.github.com/repos/uswitch/kiam/issues/94","id":444827955,"node_id":"MDEyOklzc3VlQ29tbWVudDQ0NDgyNzk1NQ==","user":{"login":"pingles","id":1150,"node_id":"MDQ6VXNlcjExNTA=","avatar_url":"https://avatars0.githubusercontent.com/u/1150?v=4","gravatar_id":"","url":"https://api.github.com/users/pingles","html_url":"https://github.com/pingles","followers_url":"https://api.github.com/users/pingles/followers","following_url":"https://api.github.com/users/pingles/following{/other_user}","gists_url":"https://api.github.com/users/pingles/gists{/gist_id}","starred_url":"https://api.github.com/users/pingles/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pingles/subscriptions","organizations_url":"https://api.github.com/users/pingles/orgs","repos_url":"https://api.github.com/users/pingles/repos","events_url":"https://api.github.com/users/pingles/events{/privacy}","received_events_url":"https://api.github.com/users/pingles/received_events","type":"User","site_admin":false},"created_at":"2018-12-06T10:39:54Z","updated_at":"2018-12-06T10:39:54Z","author_association":"CONTRIBUTOR","body":"I'm going to close this issue- the mitigation is to set the gateway parameters a little more generously.\r\n\r\n@wpalmeri I'm in the process of trying to get the last bits of documentation closed for v3 so release imminent: #195 "}}
{"actor":{"display_login":"pingles"}}