{"action":"closed","issue":{"url":"https://api.github.com/repos/deis/workflow/issues/210","repository_url":"https://api.github.com/repos/deis/workflow","labels_url":"https://api.github.com/repos/deis/workflow/issues/210/labels{/name}","comments_url":"https://api.github.com/repos/deis/workflow/issues/210/comments","events_url":"https://api.github.com/repos/deis/workflow/issues/210/events","html_url":"https://github.com/deis/workflow/issues/210","id":151931495,"number":210,"title":"Unexpected kubernetes node reboot running Deis","user":{"login":"arkkanoid","id":3811604,"avatar_url":"https://avatars.githubusercontent.com/u/3811604?v=3","gravatar_id":"","url":"https://api.github.com/users/arkkanoid","html_url":"https://github.com/arkkanoid","followers_url":"https://api.github.com/users/arkkanoid/followers","following_url":"https://api.github.com/users/arkkanoid/following{/other_user}","gists_url":"https://api.github.com/users/arkkanoid/gists{/gist_id}","starred_url":"https://api.github.com/users/arkkanoid/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/arkkanoid/subscriptions","organizations_url":"https://api.github.com/users/arkkanoid/orgs","repos_url":"https://api.github.com/users/arkkanoid/repos","events_url":"https://api.github.com/users/arkkanoid/events{/privacy}","received_events_url":"https://api.github.com/users/arkkanoid/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-04-29T17:47:40Z","updated_at":"2016-05-02T13:41:19Z","closed_at":"2016-05-02T13:41:19Z","body":"Hi,\r\nI was running Deis on AWS r3.large node. Today this node has rebooted unexpectedly. \r\nAfter reboot the node has 'Not ready' status.\r\nI've scheduled a new node and after 5 minutes it has the same 'Not ready' status.\r\nI attach you the logs previously rebooting of the second node. \r\nAfter the restart I see:\r\n\r\n    Failed Units: 1\r\n    locksmithd.service\r\n\r\nAnd no container is running.\r\n\r\nWhy is the machine rebooting?\r\n\r\n    -- Logs begin at Thu 2016-04-28 18:37:01 UTC, end at Fri 2016-04-29 14:46:40 UTC. --\r\n    Apr 29 14:37:25 ip-10-0-0-162.eu-west-1.compute.internal systemd[1]: Started docker container     e39f9fe9945e2ea236814216901956c3b5632b018be36ab58364a12308675d70.\r\n    Apr 29 14:37:25 ip-10-0-0-162.eu-west-1.compute.internal kernel: SELinux: mount invalid.  Same superblock, different security settings for (dev     mqueue, type mqueue)\r\n    Apr 29 14:37:37 ip-10-0-0-162.eu-west-1.compute.internal kubelet-wrapper[1214]: E0429 14:37:37.307728    1214 pod_workers.go:138] Error syncing pod     62b73719-0ae0-11e6-b941-068435e62e39, skipping: failed to \"StartContainer\" for \"rising-air\r\n    Apr 29 14:37:52 ip-10-0-0-162.eu-west-1.compute.internal kubelet-wrapper[1214]: E0429 14:37:52.302192    1214 pod_workers.go:138] Error syncing pod     62b73719-0ae0-11e6-b941-068435e62e39, skipping: failed to \"StartContainer\" for \"rising-air\r\n    Apr 29 14:38:05 ip-10-0-0-162.eu-west-1.compute.internal kubelet-wrapper[1214]: E0429 14:38:05.303116    1214 pod_workers.go:138] Error syncing pod     62b73719-0ae0-11e6-b941-068435e62e39, skipping: failed to \"StartContainer\" for \"rising-air\r\n    Apr 29 14:38:10 ip-10-0-0-162.eu-west-1.compute.internal kubelet-wrapper[1214]: I0429 14:38:10.851964    1214 manager.go:1748] pod \"deis-controller-    bgpmr_deis(62fceda4-0e03-11e6-b941-068435e62e39)\" container \"deis-controller\" is unhealthy\r\n    Apr 29 14:38:17 ip-10-0-0-162.eu-west-1.compute.internal kubelet-wrapper[1214]: E0429 14:38:17.306851    1214 pod_workers.go:138] Error syncing pod     62b73719-0ae0-11e6-b941-068435e62e39, skipping: failed to \"StartContainer\" for \"rising-air\r\n    Apr 29 14:38:29 ip-10-0-0-162.eu-west-1.compute.internal kubelet-wrapper[1214]: E0429 14:38:29.303075    1214 pod_workers.go:138] Error syncing pod     62b73719-0ae0-11e6-b941-068435e62e39, skipping: failed to \"StartContainer\" for \"rising-air\r\n    Apr 29 14:38:40 ip-10-0-0-162.eu-west-1.compute.internal dockerd[1003]: time=\"2016-04-29T14:38:40.852744298Z\" level=info msg=\"Container     abd8fb65712cb556db5aadb5a68dee4e4a9805389ddc840ca7a81439201e6c0e failed to exit within 30 seconds of S\r\n    Apr 29 14:38:40 ip-10-0-0-162.eu-west-1.compute.internal systemd[1]: Stopped docker container     abd8fb65712cb556db5aadb5a68dee4e4a9805389ddc840ca7a81439201e6c0e.\r\n    Apr 29 14:38:40 ip-10-0-0-162.eu-west-1.compute.internal systemd[1]: Started docker container     8a88f790b5a4794df71e3424021f462247b41c304a748e37d8e3e7e18f3e2837.\r\n    Apr 29 14:38:41 ip-10-0-0-162.eu-west-1.compute.internal kernel: SELinux: mount invalid.  Same superblock, different security settings for (dev     mqueue, type mqueue)\r\n    Apr 29 14:38:42 ip-10-0-0-162.eu-west-1.compute.internal kubelet-wrapper[1214]: E0429 14:38:42.304011    1214 pod_workers.go:138] Error syncing pod     62b73719-0ae0-11e6-b941-068435e62e39, skipping: failed to \"StartContainer\" for \"rising-air\r\n    Apr 29 14:38:56 ip-10-0-0-162.eu-west-1.compute.internal kubelet-wrapper[1214]: E0429 14:38:56.302416    1214 pod_workers.go:138] Error syncing pod     62b73719-0ae0-11e6-b941-068435e62e39, skipping: failed to \"StartContainer\" for \"rising-air\r\n    Apr 29 14:39:08 ip-10-0-0-162.eu-west-1.compute.internal kubelet-wrapper[1214]: E0429 14:39:08.302305    1214 pod_workers.go:138] Error syncing pod     62b73719-0ae0-11e6-b941-068435e62e39, skipping: failed to \"StartContainer\" for \"rising-air\r\n    Apr 29 14:39:09 ip-10-0-0-162.eu-west-1.compute.internal systemd[1]: Started docker container     756fd1ec69e999098e153847096c6347be5c227a9616e51b2ebfa6e39d6d7429.\r\n    Apr 29 14:39:10 ip-10-0-0-162.eu-west-1.compute.internal kernel: SELinux: mount invalid.  Same superblock, different security settings for (dev     mqueue, type mqueue)\r\n    Apr 29 14:39:20 ip-10-0-0-162.eu-west-1.compute.internal kubelet-wrapper[1214]: I0429 14:39:20.853085    1214 manager.go:1748] pod \"deis-controller-    bgpmr_deis(62fceda4-0e03-11e6-b941-068435e62e39)\" container \"deis-controller\" is unhealthy\r\n    Apr 29 14:39:22 ip-10-0-0-162.eu-west-1.compute.internal kubelet-wrapper[1214]: E0429 14:39:22.305683    1214 pod_workers.go:138] Error syncing pod     62b73719-0ae0-11e6-b941-068435e62e39, skipping: failed to \"StartContainer\" for \"rising-air\r\n    Apr 29 14:39:33 ip-10-0-0-162.eu-west-1.compute.internal kubelet-wrapper[1214]: E0429 14:39:33.322707    1214 pod_workers.go:138] Error syncing pod     62b73719-0ae0-11e6-b941-068435e62e39, skipping: failed to \"StartContainer\" for \"rising-air\r\n    -- Reboot --    "}}