<p>I am running load tests using K6 as a tool. I am sending 1000 requests per second to a .NET API (without database) that I deployed in Kubernetes. Sending this many requests impacts the performance of my application. This is noticeable in the request duration(as expected).</p>
<p>The K8s cluster has two nodes and one of them has high CPU usage because of all the requests I am sending (which is also expected). However, the second node has low CPU usage. While monitoring the CPU usage of the pods I am seeing the same results. Both pods use about 10% - 20% of their CPU. How can these metrics be explained? Shouldn't the loadbalancer (ingress) balance the load evenly? And why am I seeing a decrease in performance if I still have resoures left?</p>
<p>Any advice/help would be greatly appreciated.</p>
<pre><code>Node 1, CPU usage: https://i.stack.imgur.com/r0QPF.png
Node 2, CPU usage: https://i.stack.imgur.com/Ovub5.png
Pod 1, CPU usage: https://i.stack.imgur.com/ofUmm.png
Pod 2, CPU usage: https://i.stack.imgur.com/Jj3sC.png
</code></pre>

<p>Did you make sure that the pods are scheduled one on each node? Otherwise, they could be both scheduled on the same node.
For your test with one per node, a daemonset would be sufficient.
Otherwise, consider topology spread constraints:</p>
<pre><code>https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/
</code></pre>
