<p>I installed a Kubernetes Cluster on bare metal (using VMware virtual machines) with the following nodes</p>
<pre><code>master-01   Ready    control-plane,master   5d3h   v1.21.3
master-02   Ready    control-plane,master   5d3h   v1.21.3
master-03   Ready    control-plane,master   5d3h   v1.21.3
worker-01   Ready    &lt;none&gt;                 5d2h   v1.21.3
worker-02   Ready    &lt;none&gt;                 5d2h   v1.21.3
worker-03   Ready    &lt;none&gt;                 5d2h   v1.21.3
</code></pre>
<p>Metallb is installed as loadbalancer for the cluster and calico as CNI</p>
<p>I also installed nginx-ingress-controller with helm</p>
<pre><code>$ helm repo add nginx-stable https://helm.nginx.com/stable
$ helm repo update
$ helm install ingress-controller nginx-stable/nginx-ingress
</code></pre>
<p>I deployed a simple nginx server for testing</p>
<pre><code>---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx-app
  template:
    metadata:
      labels:
        app: nginx-app
    spec:
      containers:
        - name: nginx
          image: nginx
          ports:
            - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  selector:
    app: nginx-app
  #type: LoadBalancer
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-myapp
  annotations:
    # use the shared ingress-nginx
    kubernetes.io/ingress.class: &quot;nginx&quot;
spec:
  rules:
  - host: myapp.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: nginx-service
            port:
              number: 80
</code></pre>
<p>My deployments with loadbalancer types get their IP from metallb and works fine but when I add ingress although an IP is assigned I get error 502 bad gateway as shown below:</p>
<p>firewall is enabled but required ports are opened</p>
<pre><code>6443/tcp 2379-2380/tcp 10250-10252/tcp 179/tcp 7946/tcp 7946/udp 8443/tcp on master nodes
10250/tcp 30000-32767/tcp 7946/tcp 7946/udp 8443/tcp 179/tcp on worker nodes
</code></pre>
<p>My services and pods works fine</p>
<pre><code> kubectl get svc -o wide
NAME                               TYPE           CLUSTER-IP      EXTERNAL-IP    PORT(S)                      AGE   SELECTOR
ingress-controller-nginx-ingress   LoadBalancer   10.101.17.180   10.1.210.100   80:31509/TCP,443:30004/TCP   33m   app=ingress-controller-nginx-ingress
kubernetes                         ClusterIP      10.96.0.1       &lt;none&gt;         443/TCP                      6d    &lt;none&gt;
nginx-service                      ClusterIP      10.101.48.198   &lt;none&gt;         80/TCP                       31m   app=nginx-app
</code></pre>
<p>My ingress logs gives me error with no route to the internal IP</p>
<pre><code>2021/07/29 07:46:24 [error] 42#42: *8 connect() failed (113: No route to host) while connecting to upstream, client: 10.1.210.5, server: myapp.com, request: &quot;GET / HTTP/1.1&quot;, upstream: &quot;http://192.168.171.17:80/&quot;, host: &quot;myapp.com&quot;
10.1.210.5 - - [29/Jul/2021:07:46:24 +0000] &quot;GET / HTTP/1.1&quot; 502 157 &quot;-&quot; &quot;curl/7.68.0&quot; &quot;-&quot;
W0729 07:50:16.416830       1 warnings.go:70] networking.k8s.io/v1beta1 Ingress is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
192.168.2.131 - - [29/Jul/2021:07:51:03 +0000] &quot;GET / HTTP/1.1&quot; 404 555 &quot;-&quot; &quot;Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36&quot; &quot;-&quot;
192.168.2.131 - - [29/Jul/2021:07:51:03 +0000] &quot;GET /favicon.ico HTTP/1.1&quot; 404 555 &quot;http://10.1.210.100/&quot; &quot;Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36&quot; &quot;-&quot;
W0729 07:56:43.420282       1 warnings.go:70] networking.k8s.io/v1beta1 Ingress is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
W0729 08:05:28.422594       1 warnings.go:70] networking.k8s.io/v1beta1 Ingress is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
W0729 08:10:45.425329       1 warnings.go:70] networking.k8s.io/v1beta1 Ingress is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
2021/07/29 08:13:59 [error] 42#42: *12 connect() failed (113: No route to host) while connecting to upstream, client: 10.1.210.5, server: myapp.com, request: &quot;GET / HTTP/1.1&quot;, upstream: &quot;http://192.168.171.17:80/&quot;, host: &quot;myapp.com&quot;
10.1.210.5 - - [29/Jul/2021:08:13:59 +0000] &quot;GET / HTTP/1.1&quot; 502 157 &quot;-&quot; &quot;curl/7.68.0&quot; &quot;-&quot;
2021/07/29 08:14:09 [error] 42#42: *14 connect() failed (113: No route to host) while connecting to upstream, client: 10.1.210.5, server: myapp.com, request: &quot;GET / HTTP/1.1&quot;, upstream: &quot;http://192.168.171.17:80/&quot;, host: &quot;myapp.com&quot;
10.1.210.5 - - [29/Jul/2021:08:14:09 +0000] &quot;GET / HTTP/1.1&quot; 502 157 &quot;-&quot; &quot;curl/7.68.0&quot; &quot;-&quot;
</code></pre>
<p>Any idea please ?</p>
<p>EDIT : As asked here description of services and pods</p>
<pre><code>$ kubectl describe pod nginx-deployment-6f7d8d4d55-sncdr
Name:         nginx-deployment-6f7d8d4d55-sncdr
Namespace:    default
Priority:     0
Node:         worker-01/10.1.210.63
Start Time:   Thu, 29 Jul 2021 08:43:59 +0100
Labels:       app=nginx-app
              pod-template-hash=6f7d8d4d55
Annotations:  cni.projectcalico.org/podIP: 192.168.171.17/32
              cni.projectcalico.org/podIPs: 192.168.171.17/32
Status:       Running
IP:           192.168.171.17
IPs:
  IP:           192.168.171.17
Controlled By:  ReplicaSet/nginx-deployment-6f7d8d4d55
Containers:
  nginx:
    Container ID:   docker://fc61b73f8a833ad13b8956d8ce151b221b75a58a9a2fbae928464f3b0a77cca2
    Image:          nginx
    Image ID:       docker-pullable://nginx@sha256:8f335768880da6baf72b70c701002b45f4932acae8d574dedfddaf967fc3ac90
    Port:           80/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Thu, 29 Jul 2021 08:44:01 +0100
    Ready:          True
    Restart Count:  0
    Environment:    &lt;none&gt;
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-wkc48 (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  kube-api-access-wkc48:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       &lt;nil&gt;
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              &lt;none&gt;
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  16m   default-scheduler  Successfully assigned default/nginx-deployment-6f7d8d4d55-sncdr to worker-01
  Normal  Pulling    16m   kubelet            Pulling image &quot;nginx&quot;
  Normal  Pulled     16m   kubelet            Successfully pulled image &quot;nginx&quot; in 1.51808376s
  Normal  Created    16m   kubelet            Created container nginx
  Normal  Started    16m   kubelet            Started container nginx

$ kubectl describe svc ingress-controller-nginx-ingress
Name:                     ingress-controller-nginx-ingress
Namespace:                default
Labels:                   app.kubernetes.io/instance=ingress-controller
                          app.kubernetes.io/managed-by=Helm
                          app.kubernetes.io/name=ingress-controller-nginx-ingress
                          helm.sh/chart=nginx-ingress-0.10.0
Annotations:              meta.helm.sh/release-name: ingress-controller
                          meta.helm.sh/release-namespace: default
Selector:                 app=ingress-controller-nginx-ingress
Type:                     LoadBalancer
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.101.17.180
IPs:                      10.101.17.180
LoadBalancer Ingress:     10.1.210.100
Port:                     http  80/TCP
TargetPort:               80/TCP
NodePort:                 http  31509/TCP
Endpoints:                192.168.37.202:80
Port:                     https  443/TCP
TargetPort:               443/TCP
NodePort:                 https  30004/TCP
Endpoints:                192.168.37.202:443
Session Affinity:         None
External Traffic Policy:  Local
HealthCheck NodePort:     31108
Events:
  Type    Reason        Age                    From                Message
  ----    ------        ----                   ----                -------
  Normal  IPAllocated   18m                    metallb-controller  Assigned IP &quot;10.1.210.100&quot;
  Normal  nodeAssigned  3m21s (x182 over 18m)  metallb-speaker     announcing from node &quot;worker-02&quot;

$ kubectl describe svc nginx-service
Name:              nginx-service
Namespace:         default
Labels:            &lt;none&gt;
Annotations:       &lt;none&gt;
Selector:          app=nginx-app
Type:              ClusterIP
IP Family Policy:  SingleStack
IP Families:       IPv4
IP:                10.101.48.198
IPs:               10.101.48.198
Port:              &lt;unset&gt;  80/TCP
TargetPort:        80/TCP
Endpoints:         192.168.171.17:80
Session Affinity:  None
Events:            &lt;none&gt;

$ kubectl exec -it ingress-controller-nginx-ingress-dd5db86dc-gqdpm -- /bin/bash
nginx@ingress-controller-nginx-ingress-dd5db86dc-gqdpm:/$ curl 192.168.171.17:80
curl: (7) Failed to connect to 192.168.171.17 port 80: No route to host
nginx@ingress-controller-nginx-ingress-dd5db86dc-gqdpm:/$ curl 192.168.171.17
curl: (7) Failed to connect to 192.168.171.17 port 80: No route to host
nginx@ingress-controller-nginx-ingress-dd5db86dc-gqdpm:/$ curl 10.101.48.198
curl: (7) Failed to connect to 10.101.48.198 port 80: Connection timed out
nginx@ingress-controller-nginx-ingress-dd5db86dc-gqdpm:/$ curl nginx-deployment-6f7d8d4d55-sncdr
curl: (6) Could not resolve host: nginx-deployment-6f7d8d4d55-sncdr
nginx@ingress-controller-nginx-ingress-dd5db86dc-gqdpm:/$
</code></pre>
<p>To be honest I don't understand why curl svcip doesn't work anymore; yesterday it worked.</p>

tba
<p>The problem was a firewall issue I disabled firewalld and it works now, I thought that had to open port 8443 but it seems to be another port if anyone can tell me which one</p>
<p>Thank you</p>
