<p>I am thinking about paritioning my Kubernetes cluster into zones of dedicated nodes for exclusive use by dedicated sets of users as discussed <a href="https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/#example-use-cases" rel="noreferrer">here</a>. I am wondering how tainting nodes would affect <code>DaemonSets</code>, including those that are vital to cluster operation (e.g. <code>kube-proxy</code>, <code>kube-flannel-ds-amd64</code>)?</p>

<p>The <a href="https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/" rel="noreferrer">documentation</a> says daemon pods respect taints and tolerations. But if so, how can the system schedule e.g. kube-proxy pods on nodes tainted with <code>kubectl taint nodes node-x zone=zone-y:NoSchedule</code> when the pod (which is not under my control but owned by Kubernetes' own <code>DaemonSet kube-proxy</code>) does not carry a corresponding toleration.</p>

<p>What I have found empirically so far is that Kubernetes 1.14 reschedules a kube-proxy pod regardless (after I have deleted it on the tainted <code>node-x</code>), which seems to contradict the documentation. One the other hand, this does not seem to be the case for my own <code>DaemonSet</code>. When I kill its pod on <code>node-x</code> it only gets rescheduled after I remove the node's taint (or presumably after I add a toleration to the pod's spec inside the <code>DaemonSet</code>).</p>

<p>So how do <code>DaemonSet</code>s and tolerations interoperate in detail. Could it be that certain <code>DaemonSet</code>s (such as <code>kube-proxy</code>, <code>kube-flannel-ds-amd64</code>) are treated specially?</p>

rookie099
<p>Your <code>kube-proxy</code> and flannel daemonsets will have many tolerations defined in their manifest that mean they will get scheduled even on tainted nodes. </p>

<p>Here are a couple from my canal daemonset:</p>

<pre><code>tolerations:
  - effect: NoSchedule
    operator: Exists
  - key: CriticalAddonsOnly
    operator: Exists
  - effect: NoExecute
    operator: Exists
</code></pre>

<p>Here are the taints from one of my master nodes:</p>

<pre><code>taints:
  - effect: NoSchedule
    key: node-role.kubernetes.io/controlplane
    value: "true"
  - effect: NoExecute
    key: node-role.kubernetes.io/etcd
    value: "true"
</code></pre>

<p>Even though most workloads won't be scheduled on the master because of its <code>NoSchedule</code> and <code>NoExectue</code> taints, a canal pod will be run there because the daemonset <em>tolerates</em> those taints specifically.</p>

<p>The <a href="https://kubernetes.io/docs/concepts/configuration/taint-and-toleration" rel="noreferrer">doc you already linked to</a> goes into detail.</p>
