<p>I have to retype most of this by hand since the system I'm testing on can't currently connect to the internet, forgive any obvious typos please.</p>
<p>We are programmatically scheduling deployments into a large sandbox with 19 nodes, 16 of which are workers.  Usually we scan through available nodes to find the ones with the most available memory/cpu and select it for the new deployment, though given the affinity below I'm wondering if this particular deployment is being deployed through some other part of our code somehow since it has no nodeAffinity at all.</p>
<p>Either way usually deployment works, but occasionally a pod will fail to schedule</p>
<pre><code>0/19 nodes are available:  16 node(s) didn't match pod affinity rules, 16 node(s) didn't match pod affinity/anti-affinity, 3 node(s) had taint (node-role.kubernetes.io/controlplane: true), that the pod didn't tolerate
</code></pre>
<p>I've used kubectl to look up the pods affinities after they are created.  We have multiple nearly identical pods, both the ones that can be scheduled and the one that can't appear to have identical affinities:</p>
<pre><code>&quot;podAffinity&quot;: { 
   &quot;requiredDuringSchedulingIgnoreDuringExecution&quot;: [
      {
          &quot;labelSelector&quot;: {
             &quot;matchExpressions: &quot; [
                {
                    &quot;key&quot;: &quot;app.kubernetes.io/instance&quot;,
                    &quot;operator&quot;: &quot;In&quot;,
                    &quot;values&quot;: [
                       &lt;instance name&gt;
                    ]
                },
                {
                   &quot;key&quot;: &quot;host&quot;,
                   &quot;operator&quot;: &quot;In&quot;,
                   &quot;values&quot;: [
                      &quot;yes&quot;
                   ]
                 }
              ]
            },
            &quot;topologyKey&quot;: &quot;kubernetes.io/hostname&quot;
         }
      ]
   }
</code></pre>
<p>I get this by looking at spec.affinity:</p>
<pre><code>kubectl get pods &lt;pod_name&gt; -o json | jq '.spec.affinity'
</code></pre>
<p>I thought I understood affinity, but clearly not because I can't find any 'host' label on the pod or the node.  I also don't understand why the pod affinity would prevent the pod from being scheduled on a node.</p>
<p>More importantly I don't understand what a host of &quot;yes&quot; means.  It's not literally looking for a label with a value of &quot;yes&quot; is it?</p>
<p>Since I don't understand how the affinity works when assigning a functional pod I really don't understand why the same affinity occasionally fails.  I'd appreciate any help in understanding what the affinity is actually doing or why it may occasionally fail.</p>

dsollen
<p>It's about pod affinity, not node affinity. So the labels are expected to be on running pods.</p>
<p>To schedule the pod, your code requires (<code>requiredDuringSchedulingIgnoreDuringExecution</code>) that there is already a pod running on the node (<code>&quot;topologyKey&quot;: &quot;kubernetes.io/hostname&quot;</code>) that has matching labels</p>
<pre><code>apiVersion: v1
kind: Pod
metadata:
  name: foo
  labels:
    &quot;app.kubernetes.io/instance&quot;: &lt;instance-name&gt;
    host: yes
</code></pre>
<p>If such a pod is not running on one of your worker nodes, then your pod can't be scheduled.</p>
