<p>I'd like to get some clarification for preparation for maintenance when you drain nodes in a Kubernetes cluster:</p>

<p>Here's what I know when you run <code>kubectl drain MY_NODE</code>:</p>

<ul>
<li>Node is cordoned</li>
<li>Pods are gracefully shut down</li>
<li>You can opt to ignore Daemonset pods because if they are shut down, they'll just be re-spawned right away again. </li>
</ul>

<p>I'm confused as to what happens when a node is drained though.</p>

<p><strong>Questions:</strong></p>

<ul>
<li>What happens to the pods? As far as I know, there's no 'live migration' of pods in Kubernetes. </li>
<li>Will the pod be shut down and then automatically started on another node? Or does this depend on my configuration? (i.e. could a pod be shut down via drain and not start up on another node)</li>
</ul>

<p>I would appreciate some clarification on this and any best practices or advice as well. Thanks in advance.</p>

<p>By default <code>kubectl drain</code> is non-destructive, you have to override to change that behaviour. It runs with the following defaults:</p>

<blockquote>
<pre><code>  --delete-local-data=false
  --force=false
  --grace-period=-1
  --ignore-daemonsets=false
  --timeout=0s
</code></pre>
</blockquote>

<p>Each of these safeguard deals with a different category of potential destruction (local data, bare pods, graceful termination, daemonsets). It also respects pod disruption budgets to adhere to workload availability. Any non-bare pod will be recreated on a new node by its respective controller (e.g. <code>daemonset controller</code>, <code>replication controller</code>).</p>

<p>It's up to you whether you want to override that behaviour (for example you might have a bare pod if running jenkins job. If you override by setting <code>--force=true</code> it will delete that pod and it won't be recreated). If you don't override it, the node will be in drain mode indefinitely (<code>--timeout=0s</code>)).</p>
