<p>We are using AWS EKS, i deployed Promethus using the below command:</p>
<pre><code>kubectl create namespace prometheus

helm install prometheus prometheus-community/prometheus \
    --namespace prometheus \
    --set alertmanager.persistentVolume.storageClass=&quot;gp2&quot; \
    --set server.persistentVolume.storageClass=&quot;gp2&quot;
</code></pre>
<p>Once this is done i get this message:
The Prometheus server can be accessed via port 80 on the following DNS name from within your cluster:</p>
<p>The services on my prometheus deployment looks like blow:</p>
<pre><code>NAME                                    TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
service/prometheus-alertmanager         ClusterIP   10.22.210.131   &lt;none&gt;        80/TCP     20h
service/prometheus-kube-state-metrics   ClusterIP   10.12.43.248    &lt;none&gt;        8080/TCP   20h
service/prometheus-node-exporter        ClusterIP   None             &lt;none&gt;        9100/TCP   20h
service/prometheus-pushgateway          ClusterIP   10.130.54.42     &lt;none&gt;        9091/TCP   20h
service/prometheus-server               ClusterIP   10.90.94.70      &lt;none&gt;        80/TCP     20h
</code></pre>
<p>I am now using this URL in the datasource on Grafana as:</p>
<pre><code>datasources:
  datasources.yaml:
    apiVersion: 1
    datasources:
    - name: Prometheus
      type: prometheus
      url: http://prometheus-alertmanager.prometheus.svc.cluster.local
      access: proxy
      isDefault: true
</code></pre>
<p>Grafana is also up, but when the default datasource which is prometheus in this case is unable to pull any data, when i check in the datasources tab on Grafana and try to test the datasource i am getting Error reading Prometheus: client_error: client error: 404</p>
<p>since both these deployments are on the same cluster ideally it should have been able to access this.
Any help here would be highly appreciated.</p>

<p>This is because you're targeting the wrong service. You're using the alert manager url instead of the prometheus server.<br />
The URL should be this one :</p>
<pre class="lang-yaml prettyprint-override"><code>url: http://prometheus-server.prometheus.svc.cluster.local
</code></pre>
