{"action":"created","issue":{"url":"https://api.github.com/repos/kubernetes/kubernetes/issues/37194","repository_url":"https://api.github.com/repos/kubernetes/kubernetes","labels_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/37194/labels{/name}","comments_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/37194/comments","events_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/37194/events","html_url":"https://github.com/kubernetes/kubernetes/issues/37194","id":190650927,"number":37194,"title":"[k8s.io] Restart [Disruptive] should restart all nodes and ensure all nodes and pods recover","user":{"login":"Random-Liu","id":5821883,"avatar_url":"https://avatars.githubusercontent.com/u/5821883?v=3","gravatar_id":"","url":"https://api.github.com/users/Random-Liu","html_url":"https://github.com/Random-Liu","followers_url":"https://api.github.com/users/Random-Liu/followers","following_url":"https://api.github.com/users/Random-Liu/following{/other_user}","gists_url":"https://api.github.com/users/Random-Liu/gists{/gist_id}","starred_url":"https://api.github.com/users/Random-Liu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Random-Liu/subscriptions","organizations_url":"https://api.github.com/users/Random-Liu/orgs","repos_url":"https://api.github.com/users/Random-Liu/repos","events_url":"https://api.github.com/users/Random-Liu/events{/privacy}","received_events_url":"https://api.github.com/users/Random-Liu/received_events","type":"User","site_admin":false},"labels":[{"id":116712108,"url":"https://api.github.com/repos/kubernetes/kubernetes/labels/area/networking","name":"area/networking","color":"006b75","default":false},{"id":264749912,"url":"https://api.github.com/repos/kubernetes/kubernetes/labels/kind/flake","name":"kind/flake","color":"f7c6c7","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2016-11-21T08:40:38Z","updated_at":"2016-11-21T17:27:45Z","closed_at":null,"body":"https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gci-gce-serial/19\r\n\r\nThe test failed with the reason:\r\n```\r\n/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/restart.go:124\r\nExpected error:\r\n    <*errors.errorString | 0xc42068c910>: {\r\n        s: \"couldn't find 32 pods within 5m0s; last error: expected to find 32 pods but found only 29\",\r\n    }\r\n    couldn't find 32 pods within 5m0s; last error: expected to find 32 pods but found only 29\r\nnot to have occurred\r\n/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/restart.go:119\r\n```\r\n\r\nWhen check the test log, I found that before restart there are actually 32 pods:\r\n```\r\nI1118 14:16:46.689] Nov 18 14:16:46.688: INFO: Waiting up to 5m0s for pod kube-dns-3616185523-gts7f status to be running and ready, or succeeded\r\nI1118 14:16:46.689] Nov 18 14:16:46.688: INFO: Waiting up to 5m0s for pod node-problem-detector-v0.1-0ivtu status to be running and ready, or succeeded\r\nI1118 14:16:46.690] Nov 18 14:16:46.688: INFO: Waiting up to 5m0s for pod fluentd-cloud-logging-bootstrap-e2e-minion-group-e7ao status to be running and ready, or succeeded\r\nI1118 14:16:46.690] Nov 18 14:16:46.689: INFO: Waiting up to 5m0s for pod kubernetes-dashboard-3697774758-q931c status to be running and ready, or succeeded\r\nI1118 14:16:46.691] Nov 18 14:16:46.689: INFO: Waiting up to 5m0s for pod monitoring-influxdb-grafana-v4-49jz8 status to be running and ready, or succeeded\r\nI1118 14:16:46.692] Nov 18 14:16:46.689: INFO: Waiting up to 5m0s for pod kube-dns-3616185523-b0xl9 status to be running and ready, or succeeded\r\nI1118 14:16:46.692] Nov 18 14:16:46.690: INFO: Waiting up to 5m0s for pod fluentd-cloud-logging-bootstrap-e2e-master status to be running and ready, or succeeded\r\nI1118 14:16:46.692] Nov 18 14:16:46.690: INFO: Waiting up to 5m0s for pod kube-proxy-bootstrap-e2e-minion-group-f09t status to be running and ready, or succeeded\r\nI1118 14:16:46.693] Nov 18 14:16:46.690: INFO: Waiting up to 5m0s for pod kube-dns-autoscaler-2715466192-7ujy3 status to be running and ready, or succeeded\r\nI1118 14:16:46.693] Nov 18 14:16:46.690: INFO: Waiting up to 5m0s for pod e2e-image-puller-bootstrap-e2e-minion-group-f09t status to be running and ready, or succeeded\r\nI1118 14:16:46.693] Nov 18 14:16:46.690: INFO: Waiting up to 5m0s for pod kube-controller-manager-bootstrap-e2e-master status to be running and ready, or succeeded\r\nI1118 14:16:46.693] Nov 18 14:16:46.691: INFO: Waiting up to 5m0s for pod fluentd-cloud-logging-bootstrap-e2e-minion-group-f09t status to be running and ready, or succeeded\r\nI1118 14:16:46.693] Nov 18 14:16:46.691: INFO: Waiting up to 5m0s for pod kube-apiserver-bootstrap-e2e-master status to be running and ready, or succeeded\r\nI1118 14:16:46.694] Nov 18 14:16:46.691: INFO: Waiting up to 5m0s for pod etcd-server-events-bootstrap-e2e-master status to be running and ready, or succeeded\r\nI1118 14:16:46.694] Nov 18 14:16:46.691: INFO: Waiting up to 5m0s for pod kube-dns-3616185523-ru3cw status to be running and ready, or succeeded\r\nI1118 14:16:46.694] Nov 18 14:16:46.691: INFO: Waiting up to 5m0s for pod e2e-image-puller-bootstrap-e2e-minion-group-e7ao status to be running and ready, or succeeded\r\nI1118 14:16:46.694] Nov 18 14:16:46.691: INFO: Waiting up to 5m0s for pod kube-proxy-bootstrap-e2e-minion-group-ixxj status to be running and ready, or succeeded\r\nI1118 14:16:46.694] Nov 18 14:16:46.691: INFO: Waiting up to 5m0s for pod l7-lb-controller-v0.8.0-bootstrap-e2e-master status to be running and ready, or succeeded\r\nI1118 14:16:46.694] Nov 18 14:16:46.692: INFO: Waiting up to 5m0s for pod l7-default-backend-2234341178-ir0p8 status to be running and ready, or succeeded\r\nI1118 14:16:46.695] Nov 18 14:16:46.692: INFO: Waiting up to 5m0s for pod rescheduler-v0.2.1-bootstrap-e2e-master status to be running and ready, or succeeded\r\nI1118 14:16:46.695] Nov 18 14:16:46.692: INFO: Waiting up to 5m0s for pod node-problem-detector-v0.1-ly3u0 status to be running and ready, or succeeded\r\nI1118 14:16:46.695] Nov 18 14:16:46.692: INFO: Waiting up to 5m0s for pod kube-addon-manager-bootstrap-e2e-master status to be running and ready, or succeeded\r\nI1118 14:16:46.695] Nov 18 14:16:46.692: INFO: Waiting up to 5m0s for pod etcd-server-bootstrap-e2e-master status to be running and ready, or succeeded\r\nI1118 14:16:46.695] Nov 18 14:16:46.692: INFO: Waiting up to 5m0s for pod fluentd-cloud-logging-bootstrap-e2e-minion-group-ixxj status to be running and ready, or succeeded\r\nI1118 14:16:46.695] Nov 18 14:16:46.692: INFO: Waiting up to 5m0s for pod kube-scheduler-bootstrap-e2e-master status to be running and ready, or succeeded\r\nI1118 14:16:46.695] Nov 18 14:16:46.692: INFO: Waiting up to 5m0s for pod etcd-empty-dir-cleanup-bootstrap-e2e-master status to be running and ready, or succeeded\r\nI1118 14:16:46.695] Nov 18 14:16:46.692: INFO: Waiting up to 5m0s for pod kube-proxy-bootstrap-e2e-minion-group-e7ao status to be running and ready, or succeeded\r\nI1118 14:16:46.696] Nov 18 14:16:46.692: INFO: Waiting up to 5m0s for pod kube-dns-3616185523-9olcp status to be running and ready, or succeeded\r\nI1118 14:16:46.696] Nov 18 14:16:46.692: INFO: Waiting up to 5m0s for pod heapster-v1.2.0-2805816975-dll47 status to be running and ready, or succeeded\r\nI1118 14:16:46.696] Nov 18 14:16:46.692: INFO: Waiting up to 5m0s for pod e2e-image-puller-bootstrap-e2e-minion-group-ixxj status to be running and ready, or succeeded\r\nI1118 14:16:46.696] Nov 18 14:16:46.692: INFO: Waiting up to 5m0s for pod node-problem-detector-v0.1-8bu64 status to be running and ready, or succeeded\r\nI1118 14:16:46.696] Nov 18 14:16:46.692: INFO: Waiting up to 5m0s for pod node-problem-detector-v0.1-0ffdy status to be running and ready, or succeeded\r\n```\r\nHowever, after restart, there are only 29 pods:\r\n```\r\nI1118 14:24:48.528] Nov 18 14:24:48.516: INFO: e2e-image-puller-bootstrap-e2e-minion-group-e7ao       bootstrap-e2e-minion-group-e7ao  Succeeded         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:33:44 -0800 PST PodCompleted } {Ready False 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:33:55 -0800 PST PodCompleted } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:18:41 -0800 PST  }]\r\nI1118 14:24:48.528] Nov 18 14:24:48.516: INFO: e2e-image-puller-bootstrap-e2e-minion-group-f09t       bootstrap-e2e-minion-group-f09t  Succeeded         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:18:40 -0800 PST PodCompleted } {Ready False 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:18:40 -0800 PST PodCompleted }]\r\nI1118 14:24:48.528] Nov 18 14:24:48.516: INFO: e2e-image-puller-bootstrap-e2e-minion-group-ixxj       bootstrap-e2e-minion-group-ixxj  Succeeded         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:33:44 -0800 PST PodCompleted } {Ready False 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:33:54 -0800 PST PodCompleted }]\r\nI1118 14:24:48.529] Nov 18 14:24:48.516: INFO: etcd-empty-dir-cleanup-bootstrap-e2e-master            bootstrap-e2e-master             Running           [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:32:38 -0800 PST  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:33:03 -0800 PST  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:32:38 -0800 PST  }]\r\nI1118 14:24:48.529] Nov 18 14:24:48.516: INFO: etcd-server-bootstrap-e2e-master                       bootstrap-e2e-master             Running           [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:32:38 -0800 PST  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:33:02 -0800 PST  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:32:38 -0800 PST  }]\r\nI1118 14:24:48.529] Nov 18 14:24:48.516: INFO: etcd-server-events-bootstrap-e2e-master                bootstrap-e2e-master             Running           [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:32:38 -0800 PST  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:33:02 -0800 PST  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:32:38 -0800 PST  }]\r\nI1118 14:24:48.529] Nov 18 14:24:48.517: INFO: fluentd-cloud-logging-bootstrap-e2e-master             bootstrap-e2e-master             Running           [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:32:38 -0800 PST  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:33:55 -0800 PST  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:32:38 -0800 PST  }]\r\nI1118 14:24:48.530] Nov 18 14:24:48.517: INFO: fluentd-cloud-logging-bootstrap-e2e-minion-group-e7ao  bootstrap-e2e-minion-group-e7ao  Running           [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:33:44 -0800 PST  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:19:32 -0800 PST  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:18:41 -0800 PST  }]\r\nI1118 14:24:48.530] Nov 18 14:24:48.517: INFO: fluentd-cloud-logging-bootstrap-e2e-minion-group-f09t  bootstrap-e2e-minion-group-f09t  Running           [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:14:14 -0800 PST  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:19:14 -0800 PST  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:18:40 -0800 PST  }]\r\nI1118 14:24:48.530] Nov 18 14:24:48.517: INFO: fluentd-cloud-logging-bootstrap-e2e-minion-group-ixxj  bootstrap-e2e-minion-group-ixxj  Running           [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:33:44 -0800 PST  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:19:18 -0800 PST  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:18:42 -0800 PST  }]\r\nI1118 14:24:48.530] Nov 18 14:24:48.517: INFO: heapster-v1.2.0-2805816975-dll47                       bootstrap-e2e-minion-group-e7ao  Running           [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:35:22 -0800 PST  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:19:33 -0800 PST  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:35:22 -0800 PST  }]\r\nI1118 14:24:48.531] Nov 18 14:24:48.517: INFO: kube-addon-manager-bootstrap-e2e-master                bootstrap-e2e-master             Running           [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:32:38 -0800 PST  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:32:50 -0800 PST  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:32:38 -0800 PST  }]\r\nI1118 14:24:48.531] Nov 18 14:24:48.517: INFO: kube-apiserver-bootstrap-e2e-master                    bootstrap-e2e-master             Running           [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:32:38 -0800 PST  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:33:13 -0800 PST  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:32:38 -0800 PST  }]\r\nI1118 14:24:48.531] Nov 18 14:24:48.517: INFO: kube-controller-manager-bootstrap-e2e-master           bootstrap-e2e-master             Running           [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:32:38 -0800 PST  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:32:43 -0800 PST  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:32:38 -0800 PST  }]\r\nI1118 14:24:48.531] Nov 18 14:24:48.517: INFO: kube-dns-3616185523-9olcp                              bootstrap-e2e-minion-group-ixxj  Running           [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:34:29 -0800 PST  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:19:22 -0800 PST  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:34:29 -0800 PST  }]\r\nI1118 14:24:48.532] Nov 18 14:24:48.517: INFO: kube-dns-autoscaler-2715466192-7ujy3                   bootstrap-e2e-minion-group-e7ao  Running           [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:16:34 -0800 PST  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:19:25 -0800 PST  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:16:34 -0800 PST  }]\r\nI1118 14:24:48.532] Nov 18 14:24:48.517: INFO: kube-proxy-bootstrap-e2e-minion-group-e7ao             bootstrap-e2e-minion-group-e7ao  Running           [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:33:44 -0800 PST  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:18:44 -0800 PST  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:18:41 -0800 PST  }]\r\nI1118 14:24:48.532] Nov 18 14:24:48.517: INFO: kube-proxy-bootstrap-e2e-minion-group-f09t             bootstrap-e2e-minion-group-f09t  Running           [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:14:14 -0800 PST  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:18:42 -0800 PST  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:18:40 -0800 PST  }]\r\nI1118 14:24:48.533] Nov 18 14:24:48.517: INFO: kube-proxy-bootstrap-e2e-minion-group-ixxj             bootstrap-e2e-minion-group-ixxj  Running           [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:33:46 -0800 PST  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:18:45 -0800 PST  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:18:42 -0800 PST  }]\r\nI1118 14:24:48.533] Nov 18 14:24:48.517: INFO: kube-scheduler-bootstrap-e2e-master                    bootstrap-e2e-master             Running           [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:32:38 -0800 PST  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:32:44 -0800 PST  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:32:38 -0800 PST  }]\r\nI1118 14:24:48.533] Nov 18 14:24:48.517: INFO: kubernetes-dashboard-3697774758-q931c                  bootstrap-e2e-minion-group-f09t  Running           [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:16:34 -0800 PST  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:19:14 -0800 PST  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:16:34 -0800 PST  }]\r\nI1118 14:24:48.533] Nov 18 14:24:48.517: INFO: l7-default-backend-2234341178-ir0p8                    bootstrap-e2e-minion-group-ixxj  Running           [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:34:28 -0800 PST  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:19:13 -0800 PST  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:34:28 -0800 PST  }]\r\nI1118 14:24:48.534] Nov 18 14:24:48.517: INFO: l7-lb-controller-v0.8.0-bootstrap-e2e-master           bootstrap-e2e-master             Running           [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:32:38 -0800 PST  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:33:11 -0800 PST  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:32:38 -0800 PST  }]\r\nI1118 14:24:48.534] Nov 18 14:24:48.517: INFO: monitoring-influxdb-grafana-v4-49jz8                   bootstrap-e2e-minion-group-ixxj  Running           [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:34:28 -0800 PST  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:19:13 -0800 PST  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:34:28 -0800 PST  }]\r\nI1118 14:24:48.534] Nov 18 14:24:48.517: INFO: node-problem-detector-v0.1-0ffdy                       bootstrap-e2e-minion-group-f09t  Running           [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:14:14 -0800 PST  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:18:43 -0800 PST  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:14:19 -0800 PST  }]\r\nI1118 14:24:48.534] Nov 18 14:24:48.517: INFO: node-problem-detector-v0.1-0ivtu                       bootstrap-e2e-minion-group-ixxj  Running           [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:33:44 -0800 PST  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:18:45 -0800 PST  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:33:49 -0800 PST  }]\r\nI1118 14:24:48.535] Nov 18 14:24:48.517: INFO: node-problem-detector-v0.1-8bu64                       bootstrap-e2e-master             Running           [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:33:26 -0800 PST  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:33:29 -0800 PST  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:33:29 -0800 PST  }]\r\nI1118 14:24:48.535] Nov 18 14:24:48.517: INFO: node-problem-detector-v0.1-ly3u0                       bootstrap-e2e-minion-group-e7ao  Running           [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:33:44 -0800 PST  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 14:18:44 -0800 PST  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:33:50 -0800 PST  }]\r\nI1118 14:24:48.535] Nov 18 14:24:48.517: INFO: rescheduler-v0.2.1-bootstrap-e2e-master                bootstrap-e2e-master             Running           [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:32:38 -0800 PST  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:32:55 -0800 PST  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-11-18 13:32:38 -0800 PST  }]\r\n```\r\n\r\n2 kube-dns pods went away after the node restart. This seems related to the new kube-dns autoscaler. I haven't dig much, a wild guess is that the autoscaler thought that there were less nodes and scale down kubedns during the restart.\r\n\r\n@MrHohn \r\n/cc @kubernetes/sig-network "},"comment":{"url":"https://api.github.com/repos/kubernetes/kubernetes/issues/comments/262007174","html_url":"https://github.com/kubernetes/kubernetes/issues/37194#issuecomment-262007174","issue_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/37194","id":262007174,"user":{"login":"MrHohn","id":8681801,"avatar_url":"https://avatars.githubusercontent.com/u/8681801?v=3","gravatar_id":"","url":"https://api.github.com/users/MrHohn","html_url":"https://github.com/MrHohn","followers_url":"https://api.github.com/users/MrHohn/followers","following_url":"https://api.github.com/users/MrHohn/following{/other_user}","gists_url":"https://api.github.com/users/MrHohn/gists{/gist_id}","starred_url":"https://api.github.com/users/MrHohn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MrHohn/subscriptions","organizations_url":"https://api.github.com/users/MrHohn/orgs","repos_url":"https://api.github.com/users/MrHohn/repos","events_url":"https://api.github.com/users/MrHohn/events{/privacy}","received_events_url":"https://api.github.com/users/MrHohn/received_events","type":"User","site_admin":false},"created_at":"2016-11-21T17:27:45Z","updated_at":"2016-11-21T17:27:45Z","body":"This test is only run with 3 nodes. It is not possible for it to have 3 kube-dns pods.\r\n\r\nChecked the log and found this \"restart\" test started right after the \"DNS horizontal autoscaling\" test. The problem is the latter test tuned the scaling parameter for testing, and in the end it recovered the original parameter but did not wait for the additional pods to be terminated.\r\n\r\nWill send a PR to fix this soon."}}
{"actor":{"display_login":"MrHohn"}}