<p>Team,</p>

<p>We need to roll out some drivers on worker nodes of a K8s cluster and our flow is as below:</p>

<ol>
<li>cordon node [no more scheduling]</li>
<li>wait for jobs to complete</li>
<li>destroy</li>
</ol>

<p>Is there a way I can automate this using K8s options itself instead of writing some bash script to do those checks every time because we don't know when pods would complete. So, can we configure the master API server to check cordon node and destroy if it has no jobs running?</p>

<p>You can write your own application either using the <a href="https://github.com/kubernetes/client-go" rel="nofollow noreferrer">Go Client</a>, <a href="https://github.com/kubernetes-client/python" rel="nofollow noreferrer">Python Client</a>, or <a href="https://github.com/kubernetes-client/java" rel="nofollow noreferrer">Java Client</a> and basically do this:</p>

<pre><code>$ kubectl apply -f yourjob.yaml
$ kubectl cordon &lt;nodename&gt;
$ kubectl wait --for=condition=complete job/myjob
$ kubectl drain &lt;nodename&gt;
# Terminate your node if drain returns successfully
</code></pre>

<p>If this is a frequent pattern, you could also probably leverage a custom controller (<a href="https://coreos.com/operators/" rel="nofollow noreferrer">operator</a>) with a custom resource definition (<a href="https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/" rel="nofollow noreferrer">CRD</a>) to do that. You will have to embed the code of your application that talks to the API server.</p>
