<p>Imagine this hypothetical situation (that just bit me in practice):</p>

<ol>
<li>All worker instances in a Kubernetes cluster die (say due to a spot price fluctuations), and a new one comes back automatically.</li>
<li>The scheduler then attempts to schedule pods onto the node in some arbitrary order but they can't all fit because the number of nodes is smaller than before.</li>
<li>All <code>default</code> namespace pods make it on but the <code>kube-system</code> namespace DNS pod doesn't</li>
<li>Now most everything trying to run on the cluster is hung because there's no DNS on the cluster.</li>
</ol>

<p>Is there any way to use the QoS tiers in Kubernetes to get the scheduler to proritize scheduling the <code>kube-system</code> pods before the other namespaces? Or is there some other way I should be fixing this problem?</p>

<p>This is a real problem, and Kubernetes doesn't have Pod QoS guarantees yet.</p>

<p>To be completely safe, your cluster should be big enough to handle any expected cluster shrinkage, but that's not always practical.</p>

<p>At the moment, manually shrinking the competing, lower-priority deployments would probably be the easiest way to get a cluster back working.</p>

<p>There is work being done trying to get Pod QoS policies into Kubernetes. You can follow along/chime in on <a href="https://github.com/kubernetes/kubernetes/pull/14943" rel="nofollow">https://github.com/kubernetes/kubernetes/pull/14943</a></p>
