<p>My Kubernetes PODs are like a(pod_num=1) -&gt; b (pod_num=2). I create a <code>session = aiohttp.ClientSession</code> at A to connect service B using CluterIp service address(such as http://b-svc:80). When I try to add 2 pods for B, it seems that the session does not know the two newly added pods and no requests come to the newly two add pods.</p>
<p><code>aiohttp</code> has a <code>TCPConnector</code>. By default it has a 10s timeout with the dns cache. I have tried to reduce it or wait for a long time, but I found no access log to the newly added pods.</p>
<p>If I always create a new session to send requests, requests will come to the newly added pods (But it is not suggested by the aiohttp team which involves a connection pool not being used)</p>
<p>What I expect is that</p>
<blockquote>
<p>Requests automatically route [according to certain load balance rules which predefined by k8s] to the newly added pods when I <strong>reuse</strong> the client session.</p>
</blockquote>
<p>To be more specific, client at A issue requests like this:</p>
<pre class="lang-py prettyprint-override"><code>import asyncio
import aiohttp
import time

url = &quot;http://b-svc:80/v1/hello&quot;

async def main():
    session = aiohttp.ClientSession()
    for _ in range(100):
        async with session.get(url) as response:
            resp = await response.json()
            print(resp)
        time.sleep(2) # In order to make the cache invalid after 10s

if __name__ == &quot;__main__&quot;:
    asyncio.run(main())
</code></pre>
<p>Server B only returns a json response and it logs every request.</p>
<pre class="lang-json prettyprint-override"><code>{
    &quot;answer&quot;: &quot;world&quot;
}
</code></pre>
<p>At first, B has only one pod(let's say its pod index = 0) behind the ClusterIp address. When I increase the number of pods behind the ClusterIp address(Let's say, add pods with index = 1, 2, 3), only the first pod(index = 0) created earlier can receive the request [Access log only appear at pod 0, not other 1, 2, 3.]. It seems that the session does not send request to other pods.</p>

chunyang.wen
<p>In this question <a href="https://stackoverflow.com/questions/65224181/how-does-kube-proxy-handle-persistent-connections-to-a-service-between-pods">How does kube-proxy handle persistent connections to a Service between pods?</a>, the author questioned about the way that k8s handles persistent connections.</p>
<p>And from the answer and the excellent article <a href="https://learnk8s.io/kubernetes-long-lived-connections" rel="nofollow noreferrer">Load balancing and scaling long-lived connections in Kubernetes</a>, we know that k8s cannot distribute persistent connections well. In my case, by default keep alive is turned on. So those are persistent connections. If there are no new tcp connections established, no new distribution will happen. So the newly added pods will not receive the requests.</p>
<p>To disable keep alive at client will fix the problem.</p>
<pre class="lang-py prettyprint-override"><code>aiohttp.ClientSession(connector=aiohttp.TCPConnector(force_close=True))
</code></pre>
<p>Github issue:</p>
<ul>
<li><a href="https://github.com/aio-libs/aiohttp/issues/3507" rel="nofollow noreferrer">204 keep-alive client connections not handled properly</a></li>
<li><a href="https://github.com/aio-libs/aiohttp/issues/6629" rel="nofollow noreferrer">aiohttp.ClientSession together with ClusterIp not load balancing</a></li>
</ul>
