<p>I'm trying to understand the concepts of ingress and ingress controllers in kubernetes. But I'm not so sure what the end product should look like. Here is what I don't fully understand:</p>

<p>Given I'm having a running Kubernetes cluster somewhere with a master node which runes the control plane and the etcd database. Besides that I'm having like 3 worker nodes - each of the worker nodes has a public IPv4 address with a corresponding DNS A record (<code>worker{1,2,3}.domain.tld</code>) and I've full control over my DNS server. I want that my users access my web application via <code>www.domain.tld</code>. So I point the the <code>www</code> CNAME to one of the worker nodes (I saw that my ingress controller i.e. got scheduled to worker1 one so I point it to <code>worker1.domain.tld</code>).</p>

<p>Now when I schedule a workload consisting of 2 frontend pods and 1 database pod with 1 service for the frontend and 1 service for the database. From what've understand right now, I need an ingress controller pointing to the frontend service to achieve some kind of load balancing. Two questions here:</p>

<ol>
<li><p>Isn't running the ingress controller only on one worker node pointless to internally load balance two the two frontend pods via its service? Is it best practice to run an ingress controller on every worker node in the cluster?</p></li>
<li><p>For whatever reason the worker which runs the ingress controller dies and it gets rescheduled to another worker. So the ingress point will get be at another IPv4 address, right? From a user perspective which tries to access the frontend via <code>www.domain.tld</code>, this DNS entry has to be updated, right? How so? Do I need to run a specific kubernetes-aware DNS server somewhere? I don't understand the connection between the DNS server and the kubernetes cluster.</p></li>
</ol>

<p>Bonus question: If I run more ingress controllers replicas (spread across multiple workers) do I do a DNS-round robin based approach here with multiple IPv4 addresses bound to one DNS entry? Or what's the best solution to achieve HA. I rather not want to use load balancing IP addresses where the worker share the same IP address.</p>

Jens Kohl
<blockquote>
  <p>Given I'm having a running Kubernetes cluster somewhere with a master
  node which runes the control plane and the etcd database. Besides that
  I'm having like 3 worker nodes - each of the worker nodes has a public
  IPv4 address with a corresponding DNS A record
  (worker{1,2,3}.domain.tld) and I've full control over my DNS server. I
  want that my users access my web application via www.domain.tld. So I
  point the the www CNAME to one of the worker nodes (I saw that my
  ingress controller i.e. got scheduled to worker1 one so I point it to
  worker1.domain.tld).</p>
  
  <p>Now when I schedule a workload consisting of 2 frontend pods and 1
  database pod with 1 service for the frontend and 1 service for the
  database. From what've understand right now, I need an ingress
  controller pointing to the frontend service to achieve some kind of
  load balancing. Two questions here:</p>
  
  <ol>
  <li>Isn't running the ingress controller only on one worker node pointless to internally load balance two the two frontend pods via its
  service? Is it best practice to run an ingress controller on every
  worker node in the cluster?</li>
  </ol>
</blockquote>

<p>Yes, it's a good practice. Having multiple pods for the load balancer is important to ensure high availability. For example, if you run the <a href="https://github.com/kubernetes/ingress-nginx" rel="noreferrer">ingress-nginx controller</a>, you should probably deploy it to multiple nodes.</p>

<blockquote>
  <ol start="2">
  <li>For whatever reason the worker which runs the ingress controller dies and it gets rescheduled to another worker. So the ingress point
  will get be at another IPv4 address, right? From a user perspective
  which tries to access the frontend via www.domain.tld, this DNS entry
  has to be updated, right? How so? Do I need to run a specific
  kubernetes-aware DNS server somewhere? I don't understand the
  connection between the DNS server and the kubernetes cluster.</li>
  </ol>
</blockquote>

<p>Yes, the IP will change. And yes, this needs to be updated in your DNS server.</p>

<p>There are a few ways to handle this:</p>

<ol>
<li><p>assume clients will deal with outages. you can list all load balancer nodes in round-robin and assume clients will fallback. this works with some protocols, but mostly implies timeouts and problems and should generally not be used, especially since you still need to update the records by hand when k8s figures it will create/remove LB entries</p></li>
<li><p>configure an external DNS server automatically. this can be done with the <a href="https://github.com/kubernetes-incubator/external-dns" rel="noreferrer">external-dns</a> project which can sync against most of the popular DNS servers, including standard RFC2136 dynamic updates but also cloud providers like Amazon, Google, Azure, etc.</p></li>
</ol>

<blockquote>
  <p>Bonus question: If I run more ingress controllers replicas (spread
  across multiple workers) do I do a DNS-round robin based approach here
  with multiple IPv4 addresses bound to one DNS entry? Or what's the
  best solution to achieve HA. I rather not want to use load balancing
  IP addresses where the worker share the same IP address.</p>
</blockquote>

<p>Yes, you should basically do DNS round-robin. I would assume <a href="https://github.com/kubernetes-incubator/external-dns" rel="noreferrer">external-dns</a> would do the right thing here as well.</p>

<p>Another alternative is to do some sort of  <a href="https://en.wikipedia.org/wiki/Equal-cost_multi-path_routing" rel="noreferrer">ECMP</a>. This can be accomplished by having both load balancers "announce" the same IP space. That is an advanced configuration, however, which may not be necessary. There are interesting tradeoffs between BGP/ECMP and DNS updates, see <a href="https://blogs.dropbox.com/tech/2018/10/dropbox-traffic-infrastructure-edge-network/" rel="noreferrer">this dropbox engineering post</a> for a deeper discussion about those.</p>

<p>Finally, note that CoreDNS is looking at <a href="https://github.com/coredns/coredns/issues/1851" rel="noreferrer">implementing public DNS records</a> which could resolve this natively in Kubernetes, without external resources.</p>
