<p>We have an AKS test cluster with <em>four</em> Windows worker nodes and a Deployment with a replica count of <em>two</em>. The corresponding Pod spec does not specify any resource requests and limits (thus, the resulting Pods are in the BestEffort QoS class).</p>
<p>In order to conduct a performance test, we scaled all other Deployments on those worker nodes to 0 replicas and deleted all remaining Pods on the nodes. Only the system Pods created by AKS DaemonSets itself (in the <code>kube-system</code> namespace) remained. We then created the Deployment mentioned above.</p>
<p>We had assumed that the default Kubernetes scheduler would place the two replicas on different nodes by default, or at least choose nodes randomly. However, the scheduler always chose the same node to place both replicas on, no matter how often we deleted the Pods or scaled the Deployment to 0 and back again to 2. Only after we tainted that node as <code>NoSchedule</code>, did the scheduler choose another node.</p>
<p>I know I could configure anti-affinities or topology spread constraints to get a better spreading of my Pods. But in the <em>Cloud Native DevOps with Kubernetes</em> book, I read that the scheduler actually does a very good job by default and one should only use those features if absolutely necessary. (Instead maybe using the descheduler if the scheduler is forced to make bad decisions.)</p>
<p>So, I would like to understand why the behavior we observed would happen. From the <a href="https://kubernetes.io/docs/concepts/scheduling-eviction/kube-scheduler/#kube-scheduler-implementation" rel="nofollow noreferrer">docs</a>, I've learned that the scheduler first filters the nodes for fitting ones. In this case, all of them should fit, as all are configured identically. It then scores the nodes, choosing randomly if all have the same score. Why would one node always win that scoring?</p>
<p>Follow-up question: Is there some way how I could reconstruct the scheduler's decision logic in AKS? I can see <code>kube-scheduler</code> logs in Container Insights, but they don't contain any information regarding scheduling, just some operative stuff.</p>

<p>I <em>believe</em> that the scheduler is aware of which Nodes already have the container images pulled down, and will give them preference to avoid the image pull (and thus faster start time)</p>
<p>Short of digging up the source code as proof, I would guess one could create a separate Pod (for this purpose, I literally mean <code>kind: Pod</code>), force it onto one of the other Nodes via <code>nodeName:</code>, then after the Pod has been scheduled and attempted to start, delete the Pod and scale up your Deployment</p>
<p>I would then expect the new Deployment managed Pod to arrive on that other Node because it by definition has less resources in use but also has the container image required</p>
