<p>I've setup a Kubernetes 1.5 cluster with the three master nodes tainted <em>dedicated=master:NoSchedule</em>. Now I want to deploy the Nginx Ingress Controller on the Master nodes only so I've added tolerations:</p>

<pre><code>apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-ingress-controller
  namespace: kube-system
  labels:
    kubernetes.io/cluster-service: "true"
spec:
  replicas: 3
  template:
    metadata:
      labels:
        k8s-app: nginx-ingress-lb
        name: nginx-ingress-lb
      annotations:
        scheduler.alpha.kubernetes.io/tolerations: |
          [
            {
              "key": "dedicated",
              "operator": "Equal",
              "value": "master",
              "effect": "NoSchedule"
            }
          ]
    spec:
    […]
</code></pre>

<p>Unfortunately this does not have the desired effect: Kubernetes schedules all Pods on the workers. When scaling the number of replicas to a larger number the Pods are deployed on the workers, too. </p>

<p>How can I achieve scheduling to the Master nodes only?</p>

<p>Thanks for your help.</p>

<p>A toleration does <em>not</em> mean that the pod must be scheduled on a node with such taints. It means that the pod <em>tolerates</em> such a taint. If you want your pod to be <em>"attracted"</em> to specific nodes you will need to attach a <strong>label</strong> to your dedicated=master tainted nodes and set nodeSelector in the pod to look for such label.</p>

<p>Attach the label to each of your special use nodes:</p>

<pre><code>kubectl label nodes name_of_your_node dedicated=master
</code></pre>

<h2>Kubernetes 1.6 and above syntax</h2>

<p>Add the nodeSelector to your pod:</p>

<pre><code>apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: nginx-ingress-controller
  namespace: kube-system
  labels:
    kubernetes.io/cluster-service: "true"
spec:
  replicas: 3
  template:
    metadata:
      labels:
        k8s-app: nginx-ingress-lb
        name: nginx-ingress-lb
      annotations:
    spec:
      nodeSelector:
        dedicated: master
      tolerations:
      - key: dedicated
        operator: Equal
        value: master
        effect: NoSchedule
    […]
</code></pre>

<p>If you don't fancy <code>nodeSelector</code> you can add <code>affinity:</code> under <code>spec:</code> instead:</p>

<pre><code>affinity:
  nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      nodeSelectorTerms:
        matchExpressions:
        - key: dedicated
          operator: Equal
          values: ["master"]
</code></pre>

<h2>Pre 1.6 syntax</h2>

<p>Add the nodeSelector to your pod:</p>

<pre><code>apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-ingress-controller
  namespace: kube-system
  labels:
    kubernetes.io/cluster-service: "true"
spec:
  replicas: 3
  template:
    metadata:
      labels:
        k8s-app: nginx-ingress-lb
        name: nginx-ingress-lb
      annotations:
        scheduler.alpha.kubernetes.io/tolerations: |
          [
            {
              "key": "dedicated",
              "operator": "Equal",
              "value": "master",
              "effect": "NoSchedule"
            }
          ]
    spec:
      nodeSelector:
        dedicated: master
    […]
</code></pre>

<p>If you don't fancy <code>nodeSelector</code> you can also add an annotation like this:</p>

<pre><code>scheduler.alpha.kubernetes.io/affinity: &gt;
  {
    "nodeAffinity": {
      "requiredDuringSchedulingIgnoredDuringExecution": {
        "nodeSelectorTerms": [
          {
            "matchExpressions": [
              {
                "key": "dedicated",
                "operator": "Equal",
                "values": ["master"]
              }
            ]
          }
        ]
      }
    }
  }
</code></pre>

<p>Keep in mind that NoSchedule will not evict pods that are already scheduled.</p>

<p>The information above is from <a href="https://kubernetes.io/docs/user-guide/node-selection/" rel="noreferrer">https://kubernetes.io/docs/user-guide/node-selection/</a> and there are more details there.</p>
