<p>I'm a Kubernetes amateur trying to use NGINX ingress controller on GKE. I'm following <a href="https://cloud.google.com/community/tutorials/nginx-ingress-gke" rel="nofollow noreferrer">this</a> google cloud documentation to setup NGINX Ingress for my services, but, I'm facing issues in accessing the NGINX locations.</p>

<p><strong>What's working?</strong></p>

<ol>
<li>Ingress-Controller deployment using Helm (RBAC enabled)</li>
<li>ClusterIP service deployments</li>
</ol>

<p><strong>What's not working?</strong></p>

<ol>
<li>Ingress resource to expose multiple ClusterIP services using unique paths (fanout routing)</li>
</ol>

<p><strong>K8S Services</strong></p>

<pre><code>[msekar@ebs kube-base]$ kubectl get services -n payment-gateway-7682352
NAME                            TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)                      AGE
nginx-ingress-controller        LoadBalancer   10.35.241.255   35.188.161.171   80:31918/TCP,443:31360/TCP   6h
nginx-ingress-default-backend   ClusterIP      10.35.251.5     &lt;none&gt;           80/TCP                       6h
payment-gateway-dev             ClusterIP      10.35.254.167   &lt;none&gt;           5000/TCP                     6h
payment-gateway-qa              ClusterIP      10.35.253.94    &lt;none&gt;           5000/TCP                     6h
</code></pre>

<p><strong>K8S Ingress</strong></p>

<pre><code>[msekar@ebs kube-base]$ kubectl get ing -n payment-gateway-7682352
NAME                HOSTS     ADDRESS          PORTS     AGE
pgw-nginx-ingress   *         104.198.78.169   80        6h

[msekar@ebs kube-base]$ kubectl describe ing pgw-nginx-ingress -n payment-gateway-7682352
Name:             pgw-nginx-ingress
Namespace:        payment-gateway-7682352
Address:          104.198.78.169
Default backend:  default-http-backend:80 (10.32.1.4:8080)
Rules:
  Host  Path  Backends
  ----  ----  --------
  *     
        /dev/   payment-gateway-dev:5000 (&lt;none&gt;)
        /qa/    payment-gateway-qa:5000 (&lt;none&gt;)
Annotations:
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx","nginx.ingress.kubernetes.io/ssl-redirect":"false"},"name":"pgw-nginx-ingress","namespace":"payment-gateway-7682352"},"spec":{"rules":[{"http":{"paths":[{"backend":{"serviceName":"payment-gateway-dev","servicePort":5000},"path":"/dev/"},{"backend":{"serviceName":"payment-gateway-qa","servicePort":5000},"path":"/qa/"}]}}]}}

  kubernetes.io/ingress.class:               nginx
  nginx.ingress.kubernetes.io/ssl-redirect:  false
Events:                                      &lt;none&gt;
</code></pre>

<p>Last applied configuration in the annotations (ingress description output) shows the ingress resource manifest. But, I'm pasting it below for reference</p>

<pre><code>---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: pgw-nginx-ingress
  namespace: payment-gateway-7682352
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
  rules:
  - http:
      paths:
      - backend:
          serviceName: payment-gateway-dev
          servicePort: 5000
        path: /dev/
      - backend:
          serviceName: payment-gateway-qa
          servicePort: 5000
        path: /qa/
</code></pre>

<p><strong>Additional Info</strong></p>

<p>The services I'm trying to access are springboot services that use contexts, so, the root location isn't a valid end-point.</p>

<p>The container's readiness and liveliness probes are defined accordingly.</p>

<p>For example, "payment-gateway-dev" service is using a context /pgw/v1 context, so, the deployment can only be accessed through the context. To access application's swagger spec you would use the URL</p>

<p>http://&lt;>/pgw/v1/swagger-ui.html</p>

<p><strong>Behaviour of my deployment</strong></p>

<blockquote>
  <p>ingress-controller-LB-ip = 35.188.161.171</p>
</blockquote>

<ul>
<li>Accessing ingress controller load balancer "<a href="http://35.188.161.171" rel="nofollow noreferrer">http://35.188.161.171</a>" takes me to default 404 backend</li>
<li>Accessing ingress controller load balancer health "<a href="http://35.188.161.171/healthz" rel="nofollow noreferrer">http://35.188.161.171/healthz</a>" returns 200 HTTP response as expected</li>
<li>Trying to access the services using the URLs below returns "404: page not found" error

<ul>
<li><a href="http://35.188.161.171/dev/pgw/v1/swagger-ui.html" rel="nofollow noreferrer">http://35.188.161.171/dev/pgw/v1/swagger-ui.html</a></li>
<li><a href="http://35.188.161.171/qa/pgw/v1/swagger-ui.html" rel="nofollow noreferrer">http://35.188.161.171/qa/pgw/v1/swagger-ui.html</a></li>
</ul></li>
</ul>

<p>Any suggestions about or insights into what I might be doing wrong will be much appreciated.</p>

<p>+1 for this well asked question.</p>

<p>Your setup seemed right to me. In you explanation, I could find that your services would require <code>http://&lt;&gt;/pgw/v1/swagger-ui.html</code> as context. However, in your setup the path submitted to the service will be <code>http://&lt;&gt;/qa/pgw/v1/swagger-ui.html</code> if your route is <code>/qa/</code>.</p>

<p>To remove the prefix, what you would need to do is to add a <code>rewrite</code> rule to your ingress:</p>

<pre><code>apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: pgw-nginx-ingress
  namespace: payment-gateway-7682352
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
    nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
  rules:
  - http:
      paths:
      - backend:
          serviceName: payment-gateway-dev
          servicePort: 5000
        path: /dev/(.+)
      - backend:
          serviceName: payment-gateway-qa
          servicePort: 5000
        path: /qa/(.+)
</code></pre>

<p>After this, you service should receive the correct contexts.</p>

<p>Ref:</p>

<ul>
<li>Rewrite: <a href="https://github.com/kubernetes/ingress-nginx/blob/master/docs/examples/rewrite/README.md" rel="nofollow noreferrer">https://github.com/kubernetes/ingress-nginx/blob/master/docs/examples/rewrite/README.md</a></li>
<li>Ingress Route Matching: <a href="https://github.com/kubernetes/ingress-nginx/blob/master/docs/user-guide/ingress-path-matching.md" rel="nofollow noreferrer">https://github.com/kubernetes/ingress-nginx/blob/master/docs/user-guide/ingress-path-matching.md</a></li>
</ul>
