<p><s>I am facing a problem with my current k8s setup. In production, I spin up three replicas of each of our services and put them in a pod. When the pods speak to each other, we would like the pods to speak to each container in the pod in a round-robin fashion. Unfortunately, the connection between pods is never terminated thanks to TLS keep alive - and we don't want to change that part specifically - but we do want to have each container in a pod communicate properly. This is sort of what we have now:</p>

<p><a href="https://i.stack.imgur.com/DJ37b.jpg" rel="noreferrer"><img src="https://i.stack.imgur.com/DJ37b.jpg" alt="How Services Talk"></a></p>

<p>If the API is trying to talk to, say, pod OSS, it will talk to the first container only. I want API to be able to talk to all three in a round-robin fashion. </p>

<p>How do I do this? I understand that I will need an Ingress Controller, like nginx. But is there some real tutorial that breaks down how I can achieve this? I am unsure and somewhat new to k8s. Any help would be appeciated!</p>

<p>By the way, I am working locally on minikube.</p>

<p></s>
Edit:</p>

<p>In production, we spin up three replicas of each service. When service <code>A</code> needs to speak to service <code>B</code>, a pod <code>B1</code> from service <code>B</code> is selected and manages whatever request it receives. However, that pod <code>B1</code> becomes the only pod from service <code>B</code> that handles any communication; in other words, pods <code>B2</code> and <code>B3</code> are never spoken to. I am trying to solve this problem with nginx because it seems like we need a load balancer to help with this, but I'm not sure how to do it. Can anyone provide some detailed explanation on what needs to be done? Specifically, how can I set up nginx with my services so that all pods are used in a service (in some round-robin fashion), unlike what is happening now where only one pod is used? This is a problem because in production, the one pod gets overloaded with requests and dies when we have two other pods sitting there doing nothing. I'm developing locally on minikube. </p>

<p>I'm assuming that you have a microservice architecture underneath your pods, right? Have you considered the use of <a href="https://istio.io" rel="nofollow noreferrer">Istio</a> with Kubernetes? It's open sourced and developed by Google, IBM and Lyft -- intention is to give developers a vendor-neutral way (which seems to be what you are looking for) to connect, secure, manage, and monitor networks of different microservices on cloud platforms (AWS, Azure, Google, etc).</p>

<blockquote>
  <p>At a high level, Istio helps reduce the complexity of these
  deployments, and eases the strain on your development teams. It is a
  completely open source service mesh that layers transparently onto
  existing distributed applications. It is also a platform, including
  APIs that let it integrate into any logging platform, or telemetry or
  policy system. Istio’s diverse feature set lets you successfully, and
  efficiently, run a distributed microservice architecture, and provides
  a uniform way to secure, connect, and monitor microservices.</p>
</blockquote>

<p>This is the <a href="https://istio.io/docs/setup/kubernetes/multicluster-install/" rel="nofollow noreferrer">link to Istio's documentation</a>, explaining how to set up a <strong>multi cluster environment</strong> in details, which is what you are looking for.</p>

<p>There's a note in the documentation that I would like to highlight -- it may be related to your issue:</p>

<blockquote>
  <p>Since Kubernetes pods don’t have stable IPs, restart of any Istio
  service pod in the control plane cluster will cause its endpoint to be
  changed. Therefore, any connection made from remote clusters to that
  endpoint will be broken. This is documented in Istio <a href="https://github.com/istio/istio/issues/4822" rel="nofollow noreferrer">issue #4822</a>.</p>
  
  <p>There
  are a number of ways to either avoid or resolve this scenario. This
  section provides a high level overview of these options.</p>
  
  <ul>
  <li>Update the DNS entries</li>
  <li>Use a load balancer service type</li>
  <li>Expose the Istio services via a gateway</li>
  </ul>
</blockquote>

<p>I'm quoting the load balancer solution, since it seems to be what you want:</p>

<blockquote>
  <p>In Kubernetes, you can declare a service with a service type to be
  <code>LoadBalancer</code>. A simple solution to the pod restart issue is to use
  load balancers for the Istio services. You can then use the load
  balancer IPs as the Istio services’s endpoint IPs to configure the
  remote clusters.</p>
</blockquote>

<p>I hope it helps, and if you have any question, shoot!</p>
