{"action":"created","issue":{"url":"https://api.github.com/repos/sorintlab/stolon/issues/483","repository_url":"https://api.github.com/repos/sorintlab/stolon","labels_url":"https://api.github.com/repos/sorintlab/stolon/issues/483/labels{/name}","comments_url":"https://api.github.com/repos/sorintlab/stolon/issues/483/comments","events_url":"https://api.github.com/repos/sorintlab/stolon/issues/483/events","html_url":"https://github.com/sorintlab/stolon/issues/483","id":319471571,"node_id":"MDU6SXNzdWUzMTk0NzE1NzE=","number":483,"title":"Cluster stops working after turn off/on all nodes with kube api as store.","user":{"login":"sergik","id":358684,"node_id":"MDQ6VXNlcjM1ODY4NA==","avatar_url":"https://avatars1.githubusercontent.com/u/358684?v=4","gravatar_id":"","url":"https://api.github.com/users/sergik","html_url":"https://github.com/sergik","followers_url":"https://api.github.com/users/sergik/followers","following_url":"https://api.github.com/users/sergik/following{/other_user}","gists_url":"https://api.github.com/users/sergik/gists{/gist_id}","starred_url":"https://api.github.com/users/sergik/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sergik/subscriptions","organizations_url":"https://api.github.com/users/sergik/orgs","repos_url":"https://api.github.com/users/sergik/repos","events_url":"https://api.github.com/users/sergik/events{/privacy}","received_events_url":"https://api.github.com/users/sergik/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":12,"created_at":"2018-05-02T09:07:13Z","updated_at":"2018-10-24T11:46:53Z","closed_at":"2018-09-11T09:30:02Z","author_association":"NONE","body":"<!-- **NOTE:** Please submit only bug reports or RFE via the issue tracker. For other question or if unsure ask on the mailing list or gitter channel: https://github.com/sorintlab/stolon#contacts -->\r\n\r\n### Submission type\r\n\r\n<!-- Delete the inappropriate option below: -->\r\n\r\n - Bug report\r\n\r\n### Environment\r\nkube 1.9\r\n#### Stolon version\r\nlatest master tag\r\n\r\n<!-- Bug report -->\r\nWhen we faced problems with consul https://github.com/sorintlab/stolon/issues/481 we tried use built in kube api store. We turn off all our staging clusters every night. After turning cluster on we see the following error in sentinel logs and cluster stop working:\r\n\r\n```\r\n2018-05-02T08:00:28.961Z        WARN    cmd/sentinel.go:284     received db state for unexpected db uid {\"receivedDB\": \"ce6c8ee4\", \"db\": \"b5c554c0\", \"keeper\": \"172_31_57_131\"}\r\nERROR: logging before flag.Parse: E0502 08:08:11.439356       1 leaderelection.go:258] Failed to update lock: Operation cannot be fulfilled on configmaps \"stolon-cluster-postgres\": the object has been modified; please apply your changes to the latest version and try again\r\nERROR: logging before flag.Parse: E0502 08:13:46.544019       1 leaderelection.go:258] Failed to update lock: Operation cannot be fulfilled on configmaps \"stolon-cluster-postgres\": the object has been modified; please apply your changes to the latest version and try again\r\nERROR: logging before flag.Parse: E0502 08:16:16.744933       1 leaderelection.go:258] Failed to update lock: Operation cannot be fulfilled on configmaps \"stolon-cluster-postgres\": the object has been modified; please apply your changes to the latest version and try again\r\nERROR: logging before flag.Parse: E0502 08:17:31.742832       1 leaderelection.go:258] Failed to update lock: Operation cannot be fulfilled on configmaps \"stolon-cluster-postgres\": the object has been modified; please apply your changes to the latest version and try again\r\nERROR: logging before flag.Parse: E0502 08:19:15.737739       1 leaderelection.go:258] Failed to update lock: Operation cannot be fulfilled on configmaps \"stolon-cluster-postgres\": the object has been modified; please apply your changes to the latest version and try again\r\nERROR: logging before flag.Parse: E0502 08:20:21.636399       1 leaderelection.go:258] Failed to update lock: Operation cannot be fulfilled on configmaps \"stolon-cluster-postgres\": the object has been modified; please apply your changes to the latest version and try again\r\nERROR: logging before flag.Parse: E0502 08:21:05.931526       1 leaderelection.go:258] Failed to update lock: Operation cannot be fulfilled on configmaps \"stolon-cluster-postgres\": the object has been modified; please apply your changes to the latest version and try again\r\nERROR: logging before flag.Parse: E0502 08:21:43.135397       1 leaderelection.go:258] Failed to update lock: Operation cannot be fulfilled on configmaps \"stolon-cluster-postgres\": the object has been modified; please apply your changes to the latest version and try again\r\n```\r\n\r\n### Expected behaviour you didn't see\r\nStolon works correctly in case of cluster restart.\r\n### Unexpected behaviour you saw\r\nStolon crashes after turn off/on all cluster nodes.\r\n### Steps to reproduce the problem\r\n1. Use kube api as store\r\n2. Turn off/on all cluster nodes.\r\n"},"comment":{"url":"https://api.github.com/repos/sorintlab/stolon/issues/comments/432623331","html_url":"https://github.com/sorintlab/stolon/issues/483#issuecomment-432623331","issue_url":"https://api.github.com/repos/sorintlab/stolon/issues/483","id":432623331,"node_id":"MDEyOklzc3VlQ29tbWVudDQzMjYyMzMzMQ==","user":{"login":"prabhu43","id":1008923,"node_id":"MDQ6VXNlcjEwMDg5MjM=","avatar_url":"https://avatars2.githubusercontent.com/u/1008923?v=4","gravatar_id":"","url":"https://api.github.com/users/prabhu43","html_url":"https://github.com/prabhu43","followers_url":"https://api.github.com/users/prabhu43/followers","following_url":"https://api.github.com/users/prabhu43/following{/other_user}","gists_url":"https://api.github.com/users/prabhu43/gists{/gist_id}","starred_url":"https://api.github.com/users/prabhu43/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/prabhu43/subscriptions","organizations_url":"https://api.github.com/users/prabhu43/orgs","repos_url":"https://api.github.com/users/prabhu43/repos","events_url":"https://api.github.com/users/prabhu43/events{/privacy}","received_events_url":"https://api.github.com/users/prabhu43/received_events","type":"User","site_admin":false},"created_at":"2018-10-24T11:46:53Z","updated_at":"2018-10-24T11:46:53Z","author_association":"NONE","body":"@sergik What is the recommended resource limits for proxy pods with kube-api store?"}}
{"actor":{"display_login":"prabhu43"}}