<p>I am trying to setup Prometheus in k8 cluster, able to run using helm. Accessing dashboard when i expose prometheus-server as LoadBalancer service using external ip.
Same does not work when I try to configure this service as ClusterIP and making it as backend using ingress controller. Receiving 404 error, any thoughts on how to troubleshoot this?</p>

<pre><code>apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ops-ingress
  annotations:
    #nginx.org/server-snippet: "proxy_ssl_verify off;"
    nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
  rules:
  - http:
      paths:
      - path: /prometheus(/|$)(.*)
        backend:
          serviceName: prometheus-server
          servicePort: 80
</code></pre>

<p>with above ingress definition in place, url  “http://&lt;>/prometheus/ getting redirected to http://&lt;>/graph/ and then 404 error page getting rendered. When url adjusted to http://&lt;>/prometheus/graph some of webcontrols gets rendered with lots of errors on browser console.</p>

<p>Please change your Ingress configuration file, add <code>host</code> field:</p>
<pre><code>apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ops-ingress
  annotations:
    #nginx.org/server-snippet: &quot;proxy_ssl_verify off;&quot;
    nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
  rules:
  - host: prometheus.example.com
    http:
      paths:
      - path: /prometheus(/|$)(.*)
        backend:
          serviceName: prometheus-server
          servicePort: 80
</code></pre>
<p>then apply changes executing command:</p>
<pre><code>$ kubectl aply -f your_ingress_congifguration_file.yaml
</code></pre>
<p>The <code>host</code> header field in a request provides the host and port
information from the target URI, enabling the origin server to
distinguish among resources while servicing requests for multiple
host names on a single IP address.</p>
<p>Please take a look here: <a href="https://www.rfc-editor.org/rfc/rfc7230#section-5.4" rel="nofollow noreferrer">hosts-header</a>.</p>
<p>Ingress definition: <a href="https://kubernetes.io/docs/concepts/services-networking/ingress/" rel="nofollow noreferrer">ingress</a>.</p>
<p>Useful information: <a href="https://github.com/helm/charts/tree/master/stable/nginx-ingress" rel="nofollow noreferrer">helm-prometheus</a>.</p>
<p>Useful documentation: <a href="https://github.com/kubernetes/ingress-nginx/blob/351ce5394a5ec37153344135c8112f71539b5af4/docs/user-guide/ingress-path-matching.md#ingress-path-matching" rel="nofollow noreferrer">ingress-path-matching</a>.</p>
