<p>I am using K8s 1.21.0 in a bare metal configuration without a loadbalancer.
I've installed nginx ingress in a NodePort configuration using helm:</p>
<p><code>helm install ********** nginx-stable/nginx-ingress --set &quot;controller.service.type=NodePort&quot; --namespace=nginx-ingress</code></p>
<p>Edit: I've also tried this as a LoadBalancer, but the result is the same.</p>
<p>I've deployed a simple nginx web server with the following yaml:</p>
<pre><code>apiVersion: apps/v1
kind: Deployment
metadata:
  name: www-deployment
spec:
  selector:
    matchLabels:
      app: www
  replicas: 1
  template:
    metadata:
      labels:
        app: www
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80
</code></pre>
<p>and a service:</p>
<pre><code>apiVersion: v1
kind: Service
metadata:
  name: webserver-service
spec:
  selector:
    app: www
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
</code></pre>
<p>with the following ingress:</p>
<pre><code>apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: test-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: k8stest.************.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: webserver-service
            port:
              number: 80
</code></pre>
<p>From my control plane I can curl the cluster IP for the service fine. I get what I expect:</p>
<pre><code>curl 10.103.183.94
&lt;!DOCTYPE html&gt;
&lt;html&gt;
&lt;head&gt;
&lt;title&gt;Welcome to nginx!&lt;/title&gt;....
</code></pre>
<p>However, If I curl my ingress controller via its service, I always get a 404.</p>
<pre><code>curl -H &quot;Host: k8stest.**********.com&quot; http://10.106.95.61
&lt;html&gt;
&lt;head&gt;&lt;title&gt;404 Not Found&lt;/title&gt;&lt;/head&gt;
&lt;body&gt;
&lt;center&gt;&lt;h1&gt;404 Not Found&lt;/h1&gt;&lt;/center&gt;
&lt;hr&gt;&lt;center&gt;nginx/1.19.9&lt;/center&gt;
&lt;/body&gt;
&lt;/html&gt;
</code></pre>
<p>I've tried removing the rewrite target annotation, but this leads to the same result. 404.</p>
<p>It seems that the ingress controller is not communicating with my service.</p>
<p>In the ingress controller logs I am just seeing:</p>
<pre><code>[06/May/2021:08:31:53 +0000] &quot;GET / HTTP/1.1&quot; 404 153 &quot;-&quot; &quot;curl/7.68.0&quot; &quot;-&quot;
</code></pre>
<p>I've also tried with and without the host value in the ingress configuration and also tried using /test instead of just / in the ingress path. Nothing ever gets sent to the backend nginx service.</p>

<p>Ok, I ditched the helm installed nginx ingress controller and went with the bare metal installation as documented here:</p>
<p><a href="https://kubernetes.github.io/ingress-nginx/deploy/#bare-metal" rel="nofollow noreferrer">https://kubernetes.github.io/ingress-nginx/deploy/#bare-metal</a></p>
<p>As soon as this was up and running, everything sprang into life as expected.</p>
