<p>For one of my projects, I am deploying the application on Kubernetes.</p>

<p>Shortly explained, the app (in my dev machine) is deployed on 8 pods (1 / cpu core), inside a Node.
Each pod can process one request at a time (I force the limite with a threadpool - CPU is the bottleneck here).</p>

<p>I would like to force Kubernetes load balancer to send incoming request to next pod when a pod is "busy".</p>

<p>Does any of you know how to achieve this ?</p>

G. Frx
<p>You can run Kube-proxy in IPVS mode which allows routing of services based on a few different algorithms. You can find more in the <a href="https://kubernetes.io/docs/concepts/services-networking/service/#proxy-mode-ipvs" rel="nofollow noreferrer">documentation here</a>, and <a href="https://kubernetes.io/blog/2018/07/09/ipvs-based-in-cluster-load-balancing-deep-dive/" rel="nofollow noreferrer">this blog</a>. From the blog:</p>

<ul>
<li>You will need to run Kube proxy with a flag <code>--proxy-mode=ipvs</code></li>
<li><p>And add a flag <code>--ipvs-scheduler</code> for choosing one of the options</p>

<p><code>rr: round-robin</code></p>

<p><code>lc: least connection</code></p>

<p><code>dh: destination hashing</code></p>

<p><code>sh: source hashing</code></p>

<p><code>sed: shortest expected delay</code></p>

<p><code>nq: never queue</code></p></li>
</ul>

<p>As a pre-requisite the nodes will need the following modules installed:</p>

<pre><code>ip_vs
ip_vs_rr
ip_vs_wrr
ip_vs_sh
nf_conntrack_ipv4
</code></pre>

<p>If you want to do more sophisticated routing than this - then I would look into service mesh as an option</p>
