<p>I created a cluster on Google Kubernetes Engine. The nodes get deleted/created very often (at least once a day). Even though new instances are created to replace them, and pods are moved to these new nodes, I would like to understand why the nodes disappear.</p>

<p>I checked the settings used to create the cluster and the node pool:</p>

<ul>
<li>"Automatic node upgrade" is Disabled on the node pool.</li>
<li>"Pre-emptible nodes" is Disabled. </li>
<li>"Automatic node repair" is Enabled, but I doesn't look like there was a node repair, since I don't see anything in <code>gcloud container operations list</code> at the time when my nodes were deleted.</li>
</ul>

<p>I can see that the current nodes were all (re-)created at 21:00, while the cluster was created at 08:35 :</p>

<pre><code>➜  ~ gcloud container clusters describe my-cluster --format=json
{
  "createTime": "2019-04-11T08:35:39+00:00",
  ...
  "nodePools": [
    {
      ...
      "management": {
        "autoRepair": true
      },
      "name": "default-pool",
    }
  ],
  "status": "RUNNING",
  ...
}
</code></pre>

<p>How can I trace the reason why the nodes were deleted ?</p>

<p>I tried to reproduce your problem by creating a cluster, manually stopping the kubelet on a node (by running <code>systemctl stop kubelet</code>) to trigger repair and watching the node recover. In my case, I do see an operation for the auto node repair, but I can also see in the GCE operations log that the VM was deleted and recreated (by the GKE robot account). </p>

<p>If you run <code>gcloud compute operations list</code> (or check the <a href="https://pantheon.corp.google.com/compute/operations" rel="nofollow noreferrer">cloud console page</a> for operations) you should see what caused the VM to be deleted and recreated. </p>
