<p>I have deployment an application, But pod always in pending state.</p>
<pre><code>$ kubectl get nodes
NAME          STATUS   ROLES           AGE   VERSION
server1       Ready    control-plane   8d    v1.24.9
server2       Ready    worker1         8d    v1.24.9
server3       Ready    worker2         8d    v1.24.9
server4       Ready    worker3         8d    v1.24.9
</code></pre>
<pre><code>$ kubectl get all -n jenkins
NAME                          READY   STATUS    RESTARTS   AGE
pod/jenkins-6dc9f97c7-ttp64   0/1     Pending   0          7m42s
</code></pre>
<pre><code>$ kubectl describe pods jenkins-6dc9f97c7-ttp64 -n jenkins

Events:
Type     Reason            Age    From               Message
----     ------            ----   ----               -------
Warning  FailedScheduling  5m42s  default-scheduler  0/4 nodes are available: 3 node(s) had volume node affinity conflict, 4 node(s) didn't match Pod's node affinity/selector. preemption: 0/4 nodes are available: 4 Preemption is not helpful for scheduling.
</code></pre>
<p>The event history confirms a <code>FailedScheduling</code> error is the reason.</p>
<p>my <code>deployment.yml</code> has forced to assign the pod into master node.</p>
<pre><code> spec:
      nodeSelector:
        node-role.kubernetes.io/master: &quot;&quot;
      tolerations:
      - key: node-role.kubernetes.io/master
        operator: Exists
</code></pre>
<p>Since from Kubernetes version 1.20+ onwards <code>node-role.kubernetes.io/master</code> is deprecated in favor of
<code>node-role.kubernetes.io/control-plane</code> i have updated like below. However still pod showing as <code>pending</code>.</p>
<pre><code>    spec:
      nodeSelector:
        node-role.kubernetes.io/control-plane: &quot;&quot;
      tolerations:
      - key: node-role.kubernetes.io/control-plane
</code></pre>
<p><code>PersistentVolume.yml</code> side i have below content.</p>
<pre><code>...
.....
..........
  local:
    path: /ksdata/apps/nodejs/
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - server1
</code></pre>
<p><strong>Label details:-</strong></p>
<pre><code>$ kubectl get nodes --show-labels
NAME      STATUS   ROLES           AGE   VERSION   LABELS
server1   Ready    control-plane   9d    v1.24.9   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=server1,kubernetes.io/os=linux,node-role.kubernetes.io/control-plane=,node.kubernetes.io/exclude-from-external-load-balancers=
server2   Ready    worker1         9d    v1.24.9   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=server2,kubernetes.io/os=linux,node-role.kubernetes.io/worker1=worker
server3   Ready    worker2         9d    v1.24.9   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=server3,kubernetes.io/os=linux,node-role.kubernetes.io/worker2=worker
server4   Ready    worker3         9d    v1.24.9   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=server4,kubernetes.io/os=linux,node-role.kubernetes.io/worker3=worker
</code></pre>
<pre><code>$ kubectl describe node | egrep -i taint
Taints:             key=value:NoSchedule
Taints:             &lt;none&gt;
Taints:             &lt;none&gt;
Taints:             &lt;none&gt;
</code></pre>

<p>You have <strong>4</strong> nodes in the cluster, so generally one is <strong>master</strong> node, and on top of that <strong>app</strong> pods not schedule so <strong>3</strong> <strong>nodes</strong> are left.</p>
<p>While with the worker node, your deployment has of <strong>node affinity</strong> set so due to that it can't schedule a pod on that node and it's stuck in a <strong>pending</strong> state.</p>
<p>Check the <strong>PVC</strong> mostly it won't be able to get created</p>
<p><strong>Update</strong></p>
<p>Remove taint form <strong>master</strong> or <strong>control-plane</strong></p>
<pre><code>kubectl taint node server1 key=value:NoSchedule-
</code></pre>
<p><strong>Toleration to set on master</strong></p>
<pre><code>  spec:
    nodeSelector:
        kubernetes.io/hostname: &quot;server1&quot;
</code></pre>
<p>#if have tain and not removed try toleration else fine with nodeselector</p>
<pre><code>tolerations:
      - key: node-role.kubernetes.io/master
        effect: NoSchedule
</code></pre>
