<p>I'm trying to follow their docs and create this pod monitoring
i apply it and i see nothing in metrics</p>
<p>what am i doing wrong?</p>
<pre><code>apiVersion: monitoring.googleapis.com/v1
kind: ClusterPodMonitoring
metadata:
  name: monitoring
spec:
  selector:
    matchLabels:
      app: blah
  namespaceSelector:
    any: true
  endpoints:
  - port: metrics
    interval: 30s
</code></pre>

deagleshot
<p>As mentioned in the offical <a href="https://cloud.google.com/stackdriver/docs/managed-prometheus/setup-managed" rel="nofollow noreferrer">documnentation</a>:</p>
<p>The following manifest defines a PodMonitoring resource, prom-example, in the NAMESPACE_NAME namespace. The resource uses a Kubernetes label selector to find all pods in the namespace that have the label app with the value prom-example. The matching pods are scraped on a port named metrics, every 30 seconds, on the /metrics HTTP path.</p>
<pre><code>apiVersion: monitoring.googleapis.com/v1
kind: PodMonitoring
metadata:
  name: prom-example
spec:
  selector:
    matchLabels:
      app: prom-example
  endpoints:
  - port: metrics
    interval: 30s
</code></pre>
<p>To apply this resource, run the following command:</p>
<pre><code>kubectl -n NAMESPACE_NAME apply -f https://raw.githubusercontent.com/GoogleCloudPlatform/prometheus-engine/v0.5.0/examples/pod-monitoring.yaml
</code></pre>
<p>Also check the document on <a href="https://cloud.google.com/stackdriver/docs/solutions/gke/observing" rel="nofollow noreferrer">Obeserving your GKE clusters</a>.</p>
<p><strong>UPDATE:</strong></p>
<p>After applying the manifests, the managed collection will be running but no metrics will be generated. You must deploy a PodMonitoring resource that scrapes a valid metrics endpoint to see any data in the Query UI.</p>
<p>Check the logs by running the below commands:</p>
<pre><code>kubectl logs -f -ngmp-system -lapp.kubernetes.io/part-of=gmp

kubectl logs -f -ngmp-system -lapp.kubernetes.io/name=collector -c prometheus
</code></pre>
<p>If you see any error follow this <a href="https://cloud.google.com/stackdriver/docs/managed-prometheus/troubleshooting#ingest-problems" rel="nofollow noreferrer">link</a> to troubleshoot.</p>
