<p>I want to scale up/down the number of machines to increase/decrease the number of nodes in my Kubernetes cluster. When I add one machine, I’m able to successfully register it with Kubernetes; therefore, a new node is created as expected. However, it is not clear to me how to smoothly shut down the machine later. A good workflow would be:</p>

<ol>
<li>Mark the node related to the machine that I am going to shut down as unschedulable;</li>
<li>Start the pod(s) that is running in the node in other node(s);</li>
<li>Gracefully delete the pod(s) that is running in the node;</li>
<li>Delete the node.</li>
</ol>

<p>If I understood correctly, even <code>kubectl drain</code> (<a href="https://groups.google.com/forum/#!topic/google-containers/FxOYvJp82T0" rel="noreferrer">discussion</a>) doesn't do what I expect since it doesn’t start the pods before deleting them (it relies on a replication controller to start the pods afterwards which may cause downtime). Am I missing something?</p>

<p>How should I properly shutdown a machine?</p>

<p>List the nodes and get the <code>&lt;node-name&gt;</code> you want to drain or (remove from cluster) </p>

<pre><code>kubectl get nodes
</code></pre>

<p>1) First drain the node</p>

<pre><code>kubectl drain &lt;node-name&gt;
</code></pre>

<p>You might have to ignore daemonsets and local-data in the machine</p>

<pre><code>kubectl drain &lt;node-name&gt; --ignore-daemonsets --delete-local-data
</code></pre>

<p>2) Edit instance group for nodes (Only if you are using kops)</p>

<pre><code>kops edit ig nodes
</code></pre>

<p>Set the MIN and MAX size to whatever it is -1
   Just save the file (nothing extra to be done)</p>

<p>You still might see some pods in the drained node that are related to daemonsets like networking plugin, fluentd for logs, kubedns/coredns etc</p>

<p>3) Finally delete the node</p>

<pre><code>kubectl delete node &lt;node-name&gt;
</code></pre>

<p>4) Commit the state for KOPS in s3: (Only if you are using kops)</p>

<pre><code>kops update cluster --yes
</code></pre>

<p>OR (if you are using kubeadm)</p>

<p>If you are using kubeadm and would like to reset the machine to a state which was there before running <code>kubeadm join</code> then run</p>

<pre><code>kubeadm reset
</code></pre>
