<p>Can someone please help to spot the issue with <code>ingress-2</code> ingress rule? why <code>ingress-1</code> is working vs <code>ingress-2</code> is not working.</p>
<p><strong>Description of my setup, I have two deployments:</strong></p>
<p>1st deployment is of <code>nginx</code><br />
2nd deployment is of <code>httpd</code></p>
<p>Both of the deployments are exposed via <code>ClusterIP</code> services named <code>nginx-svc</code> and <code>httpd-svc</code> respectively. All the <code>endpoints</code> are proper for the services. However, while
setting up the ingress for these services, I am not able to setup the ingress using <code>host</code> (as described in <code>ingress-2</code>). however, when I am using <code>ingress-1</code>, things work fine.</p>
<p>// my host file for name resolution</p>
<pre><code>grep myapp.com /etc/hosts
127.0.0.1        myapp.com
</code></pre>
<p>// deployment details</p>
<pre><code>kubectl get deployments.apps
NAME    READY   UP-TO-DATE   AVAILABLE   AGE
nginx   3/3     3            3           29m
httpd   3/3     3            3           29m
</code></pre>
<p>// service details</p>
<pre><code>kubectl get svc
NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.152.183.1     &lt;none&gt;        443/TCP   7h48m
nginx-svc    ClusterIP   10.152.183.233   &lt;none&gt;        80/TCP    28m
httpd-svc    ClusterIP   10.152.183.58    &lt;none&gt;        80/TCP    27m
</code></pre>
<p>// endpoints details</p>
<pre><code>kubectl get ep
NAME         ENDPOINTS                                      AGE
kubernetes   10.0.2.15:16443                                7h51m
nginx-svc    10.1.198.86:80,10.1.198.87:80,10.1.198.88:80   31m
httpd-svc    10.1.198.89:80,10.1.198.90:80,10.1.198.91:80   31m
</code></pre>
<p>Attempt-1: <code>ingress-1</code></p>
<pre><code>apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-1
spec:
  rules:
  - http:
      paths:
      - path: /nginx
        pathType: Prefix
        backend:
          service:
            name: nginx-svc
            port:
              number: 80

      - path: /httpd
        pathType: Prefix
        backend:
          service:
            name: httpd-svc
            port:
              number: 80
</code></pre>
<p>// Example showing that ingress routing is working fine when <code>ingress-1</code> is used:</p>
<pre><code> curl myapp.com/nginx
&lt;!DOCTYPE html&gt;
&lt;html&gt;
&lt;head&gt;
&lt;title&gt;Welcome to nginx!&lt;/title&gt;
&lt;style&gt;
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
&lt;/style&gt;
&lt;/head&gt;
&lt;body&gt;
&lt;h1&gt;Welcome to nginx!&lt;/h1&gt;
&lt;p&gt;If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.&lt;/p&gt;

&lt;p&gt;For online documentation and support please refer to
&lt;a href=&quot;http://nginx.org/&quot;&gt;nginx.org&lt;/a&gt;.&lt;br/&gt;
Commercial support is available at
&lt;a href=&quot;http://nginx.com/&quot;&gt;nginx.com&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Thank you for using nginx.&lt;/em&gt;&lt;/p&gt;
&lt;/body&gt;
&lt;/html&gt;


 curl myapp.com/httpd
&lt;html&gt;&lt;body&gt;&lt;h1&gt;It works!&lt;/h1&gt;&lt;/body&gt;&lt;/html&gt;
</code></pre>
<p>// following ingress rule is not working as I was expecting</p>
<p>Attempt-2: <code>ingress-2</code></p>
<pre><code>kind: Ingress
metadata:
  name: ingress-2
spec:
  rules:
  - host: &quot;myapp.com&quot;
    http:
      paths:
      - pathType: Prefix
        path: &quot;/nginx&quot;
        backend:
          service:
            name: nginx-svc
            port:
              number: 80
      - pathType: Prefix
        path: &quot;/httpd&quot;
        backend:
          service:
            name: httpd-svc
            port:
              number: 80
</code></pre>
<p>// I could not spot any issue in the ing describe</p>
<pre><code>kubectl describe  ingress ingress-2
Name:             ingress-2
Namespace:        default
Address:          127.0.0.1
Default backend:  default-http-backend:80 (&lt;error: endpoints &quot;default-http-backend&quot; not found&gt;)
Rules:
  Host          Path  Backends
  ----          ----  --------
  myapp.com
                /nginx   nginx-svc:80 (10.1.198.86:80,10.1.198.87:80,10.1.198.88:80)
                /httpd   httpd-svc:80 (10.1.198.89:80,10.1.198.90:80,10.1.198.91:80)
Annotations:    &lt;none&gt;
Events:
  Type    Reason  Age                  From                      Message
  ----    ------  ----                 ----                      -------
  Normal  Sync    9m15s (x2 over 10m)  nginx-ingress-controller  Scheduled for sync
</code></pre>
<p>// example showing ingress routing is not working with this ingress resource</p>
<pre><code>curl myapp.com/nginx
&lt;html&gt;
&lt;head&gt;&lt;title&gt;404 Not Found&lt;/title&gt;&lt;/head&gt;
&lt;body&gt;
&lt;center&gt;&lt;h1&gt;404 Not Found&lt;/h1&gt;&lt;/center&gt;
&lt;hr&gt;&lt;center&gt;nginx/1.21.1&lt;/center&gt;
&lt;/body&gt;
&lt;/html&gt;

curl myapp.com/httpd
&lt;!DOCTYPE HTML PUBLIC &quot;-//IETF//DTD HTML 2.0//EN&quot;&gt;
&lt;html&gt;&lt;head&gt;
&lt;title&gt;404 Not Found&lt;/title&gt;
&lt;/head&gt;&lt;body&gt;
&lt;h1&gt;Not Found&lt;/h1&gt;
&lt;p&gt;The requested URL was not found on this server.&lt;/p&gt;
&lt;/body&gt;&lt;/html&gt;
</code></pre>

<h2 id="difference-between-ingresses">Difference between ingresses</h2>
<p>I created a one node <code>microk8s</code> cluster <a href="https://microk8s.io/" rel="noreferrer">following official documentation</a> and I wasn't able to reproduce behaviour you described which is correct behaviour. Added two pods with <code>mendhak/http-https-echo</code> image (highly recommend: very convenient for troubleshooting ingress or understanding how ingress works) and two services for each of pods.</p>
<p>The difference between two ingress rules is first ingress rule listens on all domains (HOSTS):</p>
<pre><code>$ mkctl get ing -o wide
NAME        CLASS    HOSTS   ADDRESS     PORTS   AGE
ingress-1   public   *       127.0.0.1   80      2m53s

$ curl -I --header &quot;Host: myapp.com&quot; http://127.0.0.1/httpd
HTTP/1.1 200 OK

$ curl -I --header &quot;Host: example.com&quot; http://127.0.0.1/httpd
HTTP/1.1 200 OK

$ curl -I --header &quot;Host: myapp.com&quot; http://127.0.0.1/missing_url
HTTP/1.1 404 Not Found
</code></pre>
<p>While the second ingress rule will serve only <code>myapp.com</code> domain (HOST):</p>
<pre><code>$ mkctl get ing
NAME        CLASS    HOSTS       ADDRESS     PORTS   AGE
ingress-2   public   myapp.com   127.0.0.1   80      60s

$ curl -I --header &quot;Host: myapp.com&quot; http://127.0.0.1/httpd
HTTP/1.1 200 OK

$ curl -I --header &quot;Host: example.com&quot; http://127.0.0.1/httpd
HTTP/1.1 404 Not Found
</code></pre>
<h2 id="what-exactly-happens">What exactly happens</h2>
<p>Last results in your question actually show that ingress is working as expected. You're getting responses not from <code>kubernetes ingress</code> but from pods within the cluster. First response is <code>404</code> from <code>nginx 1.21.0</code> and second is <code>404</code> from <code>apache</code>.</p>
<p>This happens because ingress sends requests to pods with the same <code>path</code> from URL without any transformations. For instance (this output I got using image mentioned above):</p>
<pre><code>$ curl myapp.com/httpd
{
  &quot;path&quot;: &quot;/httpd&quot;
...
</code></pre>
<p>While both <code>nginx</code> and <code>apache</code> are serving on <code>/</code>.</p>
<h2 id="how-to-resolve-it">How to resolve it</h2>
<p>Nginx ingress has a lot of features and one of them is <a href="https://kubernetes.github.io/ingress-nginx/examples/rewrite/" rel="noreferrer">rewriting</a> which helps to transform <code>paths</code> from what ingress gets to what goes to pods.</p>
<p>For example, if request goes to <code>http://myapp.com/nginx</code> then it will be directed to <code>nginx</code> service with <code>/nginx</code> path which will cause <code>nginx</code> to throw <code>404</code> since there's nothing on this <code>path</code>.</p>
<p>Ingress rule below fixes this by adding <code>rewrite-target</code> to <code>/</code> which we need to pass to <code>nginx</code> and <code>apache</code> services:</p>
<pre><code>apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-2
  annotations:
#    kubernetes.io/ingress.class: nginx # this should be uncommented if ingress used in &quot;regular&quot; cluster
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: myapp.com
    http:
      paths:
      - path: /nginx
        pathType: Prefix
        backend:
          service:
            name: service-a
            port:
              number: 80
      - path: /httpd
        pathType: Prefix
        backend:
          service:
            name: service-b
            port:
              number: 80
</code></pre>
<p>Quick test how it works:</p>
<pre><code>$ curl myapp.com/nginx
{
  &quot;path&quot;: &quot;/&quot;,
...
</code></pre>
<p>And</p>
<pre><code>$ curl myapp.com/httpd
{
  &quot;path&quot;: &quot;/&quot;,
...
</code></pre>
<p>As you can see now <code>path</code> is <code>/</code>.</p>
<p>Switching image to <code>nginx</code> and:</p>
<pre><code>$ curl myapp.com/nginx
&lt;!DOCTYPE html&gt;
&lt;html&gt;
&lt;head&gt;
&lt;title&gt;Welcome to nginx!&lt;/title&gt;
&lt;style&gt;
...
</code></pre>
<h2 id="useful-links">Useful links</h2>
<ul>
<li><a href="https://kubernetes.io/docs/concepts/services-networking/ingress/" rel="noreferrer">Kubernetes ingress</a></li>
<li><a href="https://kubernetes.github.io/ingress-nginx/examples/rewrite/" rel="noreferrer">Nginx ingress - rewrite</a></li>
</ul>
