<p>I'm having an issue with load balancing persistent tcp connections to my kubernetes replicas.</p>

<p>I have Unity3D clients outside of the kubernetes cluster.</p>

<p>My cluster is a baremetal cluster with metallb installed composed out of 3 nodes: 1 master and 2 workers.</p>

<p>As I have read there are two approaches:</p>

<p>1) client connects to all replicas and each time it needs to send a request it will do so on a random connection out of those that it has previously established. Periodically, it refreshes connections (in case autoscale happened or some of the persistent connections died).</p>

<p>The problem here is, I'm not sure how to access all replicas externally, headless services cannot be exposed externally.</p>

<p>2) service mesh  ?  I have vaguely read/understood that they might establish persistent tcp on your behalf. So something like this : </p>

<p>unity3d client &lt;----persistent connection ---> controller &lt;---persistent connection----> replicas</p>

<p>However, I'm not sure how to accomplish this and I'm not sure what will happen if the controller itself fails, will all the clients get their connections dropped ?  As I see it, it will come down to the same issue as the one from 1), which is allowing a client to connect to multiple different replicas at the same time with a persistent TCP connection.</p>

<p>Part of question comes as a complement to this : <a href="https://learnk8s.io/kubernetes-long-lived-connections" rel="nofollow noreferrer">https://learnk8s.io/kubernetes-long-lived-connections</a></p>

<p>In order to enable external traffic to your cluster you need an Ingress Gateway. Your ingress gateway could be <a href="https://kubernetes.io/docs/concepts/services-networking/ingress/" rel="nofollow noreferrer">the standard nginx Ingress</a>, a gateway provided by a mesh like the <a href="https://istio.io/docs/tasks/traffic-management/ingress/ingress-control/" rel="nofollow noreferrer">Istio Gateway</a> or a more specialized edge gateway like <a href="https://github.com/datawire/ambassador" rel="nofollow noreferrer">ambassador</a>, <a href="https://github.com/containous/traefik" rel="nofollow noreferrer">traefik</a>, kong, gloo, etc.</p>

<p>There are at least two ways you can perform load balancing in K8s:</p>

<ol>
<li><p>Using a <code>Service</code> resource which is just a set of iptables rules managed by the <code>kube-proxy</code> process. This is L4 load balancing only. No L7 application protocols like HTTP2 or gRPC are supported. Depending on your case, this type of LB might not be ideal for long lived connections as connections will rarely be closed.</p></li>
<li><p>Using the L7 load balancing offered by any of the <a href="https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/" rel="nofollow noreferrer">ingress controllers</a> which will skip the iptables routing (using a headless Service) and allow for more advanced load balancing algorithms.</p></li>
</ol>

<p>In order to benefit from the latter case you still need to ensure that connections are eventually terminated which is often done from the client to the proxy (while reusing connections from the proxy to the upstream). I'm not familiar with Unity3D connections but if terminating them is not an option you won't be able to do much load balancing after all.</p>

<p>When the controller fails, connections will be dropped and your client could either graciously re-attempt the connection or panic. It depends on how you code it.</p>
