<p>I have followed the instructions at <a href="https://kubernetes.github.io/ingress-nginx/user-guide/monitoring/" rel="nofollow noreferrer">https://kubernetes.github.io/ingress-nginx/user-guide/monitoring/</a> to install the Prometheus and Grafana to monitor the performance of the ingress-nginx controller.</p>
<p>However, the instructions to access the Prometheus dashboard did not work for me.</p>
<p>Indeed, according to the instructions we should be able to access the service through one of the IPs returned by the following command:</p>
<pre><code>kubectl get nodes --selector=kubernetes.io/role!=master -o jsonpath={.items[*].status.addresses[?\(@.type==\&quot;InternalIP\&quot;\)].address}
</code></pre>
<p>However, I do not understand how it is possible, since the IPs are internal to the cluster. Even in their documentation:</p>
<p><a href="https://i.stack.imgur.com/qXz3t.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/qXz3t.png" alt="enter image description here" /></a></p>
<p>How can it possibly work if the IP addresses <code>10.192.0.2 10.192.0.3 10.192.0.4</code> are internal to the cluster?</p>
<hr />
<p>Here is what I have:</p>
<pre><code>mark@D-LP0003:~$ kubectl get svc -n ingress-nginx
NAME                                 TYPE           CLUSTER-IP     EXTERNAL-IP     PORT(S)                      AGE
grafana                              NodePort       10.0.6.223     &lt;none&gt;          3000:32275/TCP               23m
ingress-nginx-controller             LoadBalancer   10.0.254.117   40.76.145.210   80:32542/TCP,443:31960/TCP   51m
ingress-nginx-controller-admission   ClusterIP      10.0.110.16    &lt;none&gt;          443/TCP                      51m
prometheus-server                    NodePort       10.0.208.16    &lt;none&gt;          9090:32053/TCP               48m
mark@D-LP0003:~$
</code></pre>
<p>No external IP. OK, the documentation has it in mind already and suggests to run another command:</p>
<pre><code>mark@D-LP0003:~$ kubectl get nodes -o wide
NAME                                STATUS   ROLES   AGE   VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION      CONTAINER-RUNTIME
aks-nodepool1-30294001-vmss000000   Ready    agent   14h   v1.17.9   10.240.0.4    &lt;none&gt;        Ubuntu 16.04.7 LTS   4.15.0-1092-azure   docker://3.0.10+azure
aks-nodepool1-30294001-vmss000001   Ready    agent   14h   v1.17.9   10.240.0.5    &lt;none&gt;        Ubuntu 16.04.7 LTS   4.15.0-1092-azure   docker://3.0.10+azure
aks-nodepool1-30294001-vmss000003   Ready    agent   11h   v1.17.9   10.240.0.6    &lt;none&gt;        Ubuntu 16.04.7 LTS   4.15.0-1092-azure   docker://3.0.10+azure
mark@D-LP0003:~$
</code></pre>
<p>Still no external IP. And at this point the documentation suggests to run yet another command:</p>
<pre><code>mark@D-LP0003:~$ kubectl get nodes --selector=kubernetes.io/role!=master -o jsonpath={.items[*].status.addresses[?\(@.type==\&quot;InternalIP\&quot;\)].address}
10.240.0.4 10.240.0.5 10.240.0.6mark@D-LP0003:~$
</code></pre>
<p>I get back the same internal IPs.</p>
<p>The only way I found to access the dashboard is through the client side port forwarding:</p>
<pre><code>kubectl port-forward svc/prometheus-server 9090:9090 -n ingress-nginx  &amp;
</code></pre>
<p>And then I can access the server at <code>http://localhost:9090</code>.</p>
<p>But I do not want it. The service is supposed to be externally accessible, so how do I access it?</p>

mark
<p>You have few options</p>
<ol>
<li><p>Since you are in Azure you could create a <code>LoadBalancer</code> type service instead of <code>NodePort</code> type service to expose grafana and prometheus. You will get an external IP for the service which you can use to access it.</p>
</li>
<li><p>Use the Master nodes IP and NodePort to access it. Assuming master nodes have external IP. Use <code>MASTERNODEIP:32053</code> for prometheus and <code>MASTERNODEIP:32275</code> for grafana</p>
</li>
<li><p>Use ingress resource to expose them. You already have a external IP for the LoadBalancer of the nginx ingress controller. Use that IP to access it. Refer to <a href="https://stackoverflow.com/questions/59537655/prometheus-dashboard-exposed-over-ingress-controller">this</a> to know how to do it.</p>
</li>
</ol>
