<p>I'm trying to wrap my head around exposing internal loadbalancing to outside world on bare metal k8s cluster.</p>
<p>Let's say we have a basic cluster:</p>
<ol>
<li><p>Some master nodes and some worker nodes, that has two interfaces, one public facing (eth0) and one local(eth1) with ip within 192.168.0.0/16 network</p>
</li>
<li><p>Deployed MetalLB and configured 192.168.200.200-192.168.200.254 range for its internal ips</p>
</li>
<li><p>Ingress controller with its service with type LoadBalancer</p>
</li>
</ol>
<p>MetalLB now should assign one of the ips from 192.168.200.200-192.168.200.254 to ingress service, as of my current understanding.</p>
<p>But I have some following questions:</p>
<p>On every node I could curl ingress controller externalIP (as long as they are reachable on eth1) with host header attached and get a response from a service thats configured in coresponding ingress resource or is it valid only on node where Ingress pods are currently placed?</p>
<p>What are my options to pass incoming <strong>external</strong> traffic to eth0 to an ingress listening on eth1 network?</p>
<p>Is it possible to forward requests saving source ip address or attaching X-Forwarded-For header is the only option?</p>

Alex Smith
<p>Assuming that we are talking about <code>Metallb</code> using <code>Layer2</code>.</p>
<p>Addressing the following questions:</p>
<blockquote>
<p>On every node I could curl ingress controller externalIP (as long as they are reachable on eth1) with host header attached and get a response from a service thats configured in coresponding ingress resource or is it valid only on node where Ingress pods are currently placed?</p>
</blockquote>
<blockquote>
<p>Is it possible to forward requests saving source ip address or attaching X-Forwarded-For header is the only option?</p>
</blockquote>
<p>Dividing the solution on the premise of preserving the source IP, this question could go both ways:</p>
<hr />
<h3>Preserve the source IP address</h3>
<p>To do that you would need to set the <code>Service of type LoadBalancer</code> of your <code>Ingress controller</code> to support &quot;Local traffic policy&quot; by setting (in your <code>YAML</code> manifest):</p>
<ul>
<li><code>.spec.externalTrafficPolicy: Local</code></li>
</ul>
<p>This setup will be valid as long as on each <code>Node</code> there is replica of your <code>Ingress controller</code> as all of the networking coming to your controller will be contained in a single <code>Node</code>.</p>
<p>Citing the official docs:</p>
<blockquote>
<p>With the <code>Local</code> traffic policy, <code>kube-proxy</code> on the node that received the traffic sends it only to the service’s pod(s) that are on the same node. There is no “horizontal” traffic flow between nodes.</p>
<p>Because <code>kube-proxy</code> doesn’t need to send traffic between cluster nodes, your pods can see the real source IP address of incoming connections.</p>
<p>The downside of this policy is that incoming traffic only goes to some pods in the service. Pods that aren’t on the current leader node receive no traffic, they are just there as replicas in case a failover is needed.</p>
<p><em><a href="https://metallb.universe.tf/usage/#local-traffic-policy" rel="nofollow noreferrer">Metallb.universe.tf: Usage: Local traffic policy</a></em></p>
</blockquote>
<hr />
<h3>Do not preserve the source IP address</h3>
<p>If your use case does not require you to preserve the source IP address, you could go with the:</p>
<ul>
<li><code>.spec.externalTrafficPolicy: Cluster</code></li>
</ul>
<p>This setup won't require that the replicas of your <code>Ingress controller</code> will be present on each <code>Node</code>.</p>
<p>Citing the official docs:</p>
<blockquote>
<p>With the default <code>Cluster</code> traffic policy, <code>kube-proxy</code> on the node that received the traffic does load-balancing, and distributes the traffic to all the pods in your service.</p>
<p>This policy results in uniform traffic distribution across all pods in the service. However, <code>kube-proxy</code> will obscure the source IP address of the connection when it does load-balancing, so your pod logs will show that external traffic appears to be coming from the service’s leader node.</p>
<p><em><a href="https://metallb.universe.tf/usage/#cluster-traffic-policy" rel="nofollow noreferrer">Metallb.universe.tf: Usage: Cluster traffic policy</a></em></p>
</blockquote>
<hr />
<p>Addressing the 2nd question:</p>
<blockquote>
<p>What are my options to pass incoming external traffic to eth0 to an ingress listening on eth1 network?</p>
</blockquote>
<p>Metallb listen by default on all interfaces, all you need to do is to specify the address pool from this <code>eth</code> within Metallb config.</p>
<p>You can find more reference on this topic by following:</p>
<ul>
<li><em><a href="https://metallb.universe.tf/faq/#in-layer-2-mode-how-to-specify-the-host-interface-for-an-address-pool" rel="nofollow noreferrer">Metallb.universe.tf: FAQ: In layer 2 mode how to specify the host interface for an address pool</a></em></li>
</ul>
<p>An example of such configuration, could be following:</p>
<pre class="lang-yaml prettyprint-override"><code>apiVersion: v1
kind: ConfigMap
metadata:
  namespace: metallb-system
  name: config
data:
  config: |
    address-pools: # HERE
    - name: my-ip-space
      protocol: layer2
      addresses:
      - 192.168.1.240/28
</code></pre>
