<p>I am using aws EKS with a managed node group. Twice in the passed couple of weeks I had a case where the Kubelet in one of the nodes crashed or stopped reporting back to the control plane.</p>
<p>In this case I would expect the Autoscaling group to identify this node as unhealthy, and replace it. However, this is not what happens. I have recreated the issue by creating a node and manually stopping the Kubelet, see image  below:</p>
<p><a href="https://i.stack.imgur.com/nleip.png" rel="noreferrer"><img src="https://i.stack.imgur.com/nleip.png" alt="enter image description here" /></a></p>
<p>My first thought was to create an Event Bus alert that would trigger a lambda to take care of this but I couldn't find the EKS service in the list of services in Event Bus, so …</p>
<p>Does anyone know of a tool or configuration that would help with this?
To be clear I am looking for something that would:</p>
<ol>
<li>Detect that that kubelet isn't connecting to the control plane</li>
<li>Delete the node in the cluster</li>
<li>Terminate the EC2</li>
</ol>
<p>THANKS!!</p>

<p>I would suggest looking at the <a href="https://github.com/kubernetes/node-problem-detector" rel="nofollow noreferrer">node-problem-detector</a> or this <a href="https://blog.cloudflare.com/automatic-remediation-of-kubernetes-nodes/" rel="nofollow noreferrer">blog</a> by Cloudflare. There is an <a href="https://github.com/aws/containers-roadmap/issues/928" rel="nofollow noreferrer">issue</a> on the EKS roadmap for automated node health checking. I would upvote the issue if it's important to you.</p>
