<p>I have a ready-made Kubernetes cluster with configured grafana + prometheus(operator) monitoring.
I added the following labels to pods with my app:</p>
<pre class="lang-yaml prettyprint-override"><code>prometheus.io/scrape: &quot;true&quot;
prometheus.io/path: &quot;/my/app/metrics&quot;
prometheus.io/port: &quot;80&quot;
</code></pre>
<p>But metrics don't get into Prometheus. However, prometheus has all the default Kubernetes metrics.</p>
<p>What is the problem?</p>

<p>You should create <a href="https://github.com/prometheus-operator/prometheus-operator/blob/master/Documentation/user-guides/getting-started.md#include-servicemonitors" rel="nofollow noreferrer"><code>ServiceMonitor</code> or <code>PodMonitor</code> objects</a>.</p>
<p><code>ServiceMonitor</code> which describes the set of targets to be monitored by Prometheus. The Operator automatically generates Prometheus scrape configuration based on the definition and the targets will have the IPs of all the pods behind the service.</p>
<p>Example:</p>
<pre><code>apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: example-app
  labels:
    team: frontend
spec:
  selector:
    matchLabels:
      app: example-app
  endpoints:
  - port: web
</code></pre>
<p><code>PodMonitor</code>, which declaratively specifies how groups of pods should be monitored. The Operator automatically generates Prometheus scrape configuration based on the definition.</p>
<p>Example:</p>
<pre><code>apiVersion: monitoring.coreos.com/v1
kind: PodMonitor
metadata:
  name: example-app
  labels:
    team: frontend
spec:
  selector:
    matchLabels:
      app: example-app
  podMetricsEndpoints:
  - port: web
</code></pre>
