<p>I have total <strong>3</strong> node pools are as follow:</p>
<ol>
<li><strong>database pool</strong> - regular node pool</li>
<li><strong>content pool</strong> - regular node pool</li>
<li><strong>content spot pool</strong> - spot node pool</li>
</ol>
<p>Initially, <strong>content pool</strong> have <strong>0 node</strong> count with enabled autoscaler.
I have deployed one nginx pod deployment on the <strong>content spot pool</strong>. which has minimum node count 1 and maximum node count 3.
The deployment file for nginx are as follow:</p>
<pre><code>apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80
      affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 2
            preference:
              matchExpressions:
              - key: agentpool
                operator: In
                values:
                - contentspotpool
          - weight: 1
            preference:
              matchExpressions:
              - key: agentpool
                operator: In
                values:
                - contentpool
</code></pre>
<p>When the content spot pool is evicted I want that the pod on the <strong>content spot pool</strong> are to be shifted on <strong>content pool</strong>. But, the pod are scheduled on the <strong>database pool</strong>..!</p>
<p>Can anyone tell me how I can achieve this?..</p>
<p>Also How can I setup a <strong>database pool</strong> in such way that it refuses all the new pods?</p>
<p>AKS version used - 1.18.14</p>

Kaivalya Dambalkar
<p>I decided to provide a Community Wiki answer as there was a similar answer but was deleted by its author.</p>
<p>In this case, you can use <a href="https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/" rel="nofollow noreferrer">Taints and Tolerations</a>:</p>
<blockquote>
<p>Taints and tolerations work together to ensure that pods are not scheduled onto inappropriate nodes. One or more taints are applied to a node; this marks that the node should not accept any pods that do not tolerate the taints.</p>
</blockquote>
<p>You may add a <code>taint</code> to nodes from the <strong>database pool</strong> and specify a <code>toleration</code> that &quot;matches&quot; this <code>taint</code> only for Pods that can be scheduled on the <strong>database pool</strong>.</p>
<hr />
<p>I've created a simple example to illustrate how it may work.</p>
<p>I have only one worker node and I added a specific <code>taint</code> to this node:</p>
<pre><code>$ kubectl get nodes
NAME                STATUS   ROLES    AGE     
database-pool       Ready    &lt;none&gt;   6m9s  

$ kubectl taint nodes database-pool type=database:NoSchedule
node/database-pool tainted

$ kubectl describe node database-pool | grep -i taint
Taints:             type=database:NoSchedule
</code></pre>
<p>Only <code>Pods</code> with the following <code>toleration</code> will be allowed to be scheduled onto the <code>database-pool</code> node:</p>
<pre><code>tolerations:
- key: &quot;type&quot;
  operator: &quot;Equal&quot;
  value: &quot;database&quot;
  effect: &quot;NoSchedule&quot;
</code></pre>
<p>I created two <code>Pods</code>: <code>web</code> (does not tolerate the <code>taint</code>) and <code>web-with-toleration</code> (tolerates the <code>taint</code>):</p>
<pre><code>$ cat web.yml
apiVersion: v1
kind: Pod
metadata:
  labels:
    run: web
  name: web
spec:
  containers:
  - image: nginx
    name: web
    
$ cat web-with-toleration.yml
apiVersion: v1
kind: Pod
metadata:
  labels:
    run: web
  name: web-with-toleration
spec:
  containers:
  - image: nginx
    name: web
  tolerations:
  - key: &quot;type&quot;
    operator: &quot;Equal&quot;
    value: &quot;database&quot;
    effect: &quot;NoSchedule&quot;

$ kubectl apply -f web.yml
pod/web created
$ kubectl apply -f web-with-toleration.yml
pod/web-with-toleration created
</code></pre>
<p>Finally, we can check which <code>Pod</code> has been correctly scheduled:</p>
<pre><code>$ kubectl get pods -o wide
NAME                  READY   STATUS    RESTARTS   AGE     IP           NODE                                         
web                   0/1     Pending   0          6m13s   &lt;none&gt;       &lt;none&gt;                                       
web-with-toleration   1/1     Running   0          6m8s    10.76.0.14   database-pool 
</code></pre>
<p><br>It is possible to use <a href="https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity" rel="nofollow noreferrer"><em>Node affinity</em></a> and <em>Taints</em> at the same time to have a great control over the placement of <code>Pods</code> on specific nodes.</p>
<blockquote>
<p>Node affinity, is a property of Pods that attracts them to a set of nodes (either as a preference or a hard requirement).
Taints are the opposite -- they allow a node to repel a set of pods.</p>
</blockquote>
