<p>I have a DigitalOcean Kubernetes cluster. I have installed the NGINX Ingress Controller via Helm &amp; also installed Prometheus &amp; Grafana.</p>

<p>My ingresses are in the <code>default</code> namespace, my monitoring is in a <code>monitoring</code> namespace.</p>

<p>Here are the versions of the charts i have installed.</p>

<pre><code>❯ helm list
NAME            REVISION    UPDATED                     STATUS      CHART                   APP VERSION NAMESPACE
grafana         1           Mon Oct  7 08:04:15 2019    DEPLOYED    grafana-3.8.18          6.3.5       monitoring
metrics-server  1           Thu Aug 29 09:07:21 2019    DEPLOYED    metrics-server-2.8.2    0.3.2       kube-system
nginx-ingress   1           Wed Aug 21 21:32:06 2019    DEPLOYED    nginx-ingress-1.17.1    0.25.1      default
prometheus      1           Mon Oct  7 09:24:21 2019    DEPLOYED    prometheus-9.1.2        2.11.1      monitoring
</code></pre>

<p>I'm trying to get some NGINX Metrics, so i can monitor in Grafana.</p>

<p>However, none of them appear in the Prometheus UI. I have tried adding the prometheus flags to all my ingresses, but still get nothing. e.g.</p>

<pre><code>apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ****-ingress
  namespace: monitoring
  annotations:
    kubernetes.io/ingress.class: nginx
    enable-vts-status: "true"
    prometheus.io/scrape: "true"
    prometheus.io/port: "10254"
</code></pre>

<p>I used the default <a href="https://github.com/helm/charts/blob/master/stable/nginx-ingress/values.yaml" rel="nofollow noreferrer">values.yaml</a> file for the nginx controller, but i did change to enable metrics:</p>

<pre><code>  metrics:
    enabled: true
</code></pre>

<p>From what i read, it should work out of the box. So i have no idea what's going wrong.</p>

<p><a href="https://i.stack.imgur.com/sQjOP.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/sQjOP.png" alt="Prometheus Service Discovery"></a>
<a href="https://i.stack.imgur.com/JuOb1.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/JuOb1.png" alt="Prometheus Search Query"></a></p>

<p>One thing that does look suspicious, is that the service discovery doesn't seem to be monitoring any services, but i've never used Prometheus and i'm at a dead end with what to look for.</p>

<p>Thank you</p>

<p>I got there in the end. I have to create a prometheus job, to watch the NGINX Metrics server. I figured, that difference between my setup and all the examples online (even though i used the same helm charts) is that i have a separate NGINX metrics service, running on a different port. So OOB Prometheus was not configured to check it.</p>

<pre><code>- job_name: nginx-ingress
      metrics_path: /metrics
      scrape_interval: 5s
      static_configs:
        - targets:
          - nginx-ingress-controller-metrics:9913
</code></pre>

<p>Added that to the prometheus configmap, and reloaded the config. </p>
