<p>We have a Kubernetes Pod which provides multiple metric endpoints (<code>:3093/metrics</code> and <code>:9113/metrics</code>), but it also has an port which doesn't provide any metric (<code>:80</code>).</p>

<p><strong>TL;DR:</strong> Is is possible to scrape only the ports <code>3093</code> and <code>9113</code>?</p>

<hr>

<p>We are using <a href="https://github.com/prometheus/prometheus/blob/70f3d1e/documentation/examples/prometheus-kubernetes.yml" rel="nofollow noreferrer">the example configuration</a> for the scrape configuration:</p>



<pre><code>- job_name: 'kubernetes-pods'
  tls_config:
    insecure_skip_verify: true
  kubernetes_sd_configs:
  - role: pod
  relabel_configs:
  - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
    action: keep
    regex: true
  - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
    action: replace
    target_label: __metrics_path__
    regex: (.+)
  - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
    action: replace
    regex: ([^:]+)(?::\d+)?;(\d+)
    replacement: $1:$2
    target_label: __address__
  - action: labelmap
    regex: __meta_kubernetes_pod_label_(.+)
  - source_labels: [__meta_kubernetes_namespace]
    action: replace
    target_label: kubernetes_namespace
  - source_labels: [__meta_kubernetes_pod_name]
    action: replace
    target_label: kubernetes_pod_name
</code></pre>

<p>These endpoints get properly scraped, when enabling scraping with this annotation:</p>

<pre><code>  annotations:
    prometheus.io/scrape: "true"
</code></pre>

<p>But this has the issue, that it also scrapes port <code>:80</code>, which it shouldn't.</p>

<p>The suggestion outlined <a href="https://github.com/prometheus/prometheus/issues/3756#issuecomment-767147462" rel="nofollow noreferrer">here</a> allows scraping multiple endpoints per pod. The idea is to give custom names to container ports, which must be scraped, so these names can be used later during relabeling phase for the discovered targets. For example, the following config adds <code>prom-metrics</code> name to ports 3093 and 9113, which must be scraped by Prometheus. Note that the port 80 has no <code>prom-metrics</code> name, so it won't be scraped.</p>
<pre class="lang-yaml prettyprint-override"><code>containers:
- name: custom-app
  image: github.com/user/app:latest
  ports:
  - containerPort: 80
  - containerPort: 3093
    name: prom-metrics
  - containerPort: 9113
    name: prom-metrics
</code></pre>
<p>Later the following scrape config can be used for scraping container ports with <code>prom-metrics</code> name:</p>
<pre class="lang-yaml prettyprint-override"><code>  - job_name: 'pods-with-prom-metrics'
    kubernetes_sd_configs:
    - role: pod
    relabel_configs:
    - source_labels: [__meta_kubernetes_pod_container_port_name]
      regex: prom-metrics
      action: keep
    - source_labels: [__address__, __meta_kubernetes_pod_container_port_number]
      regex: ([^:]+)(?::\d+)?;(\d+)
      replacement: $1:$2
      target_label: __address__
    - action: labelmap
      regex: __meta_kubernetes_pod_label_(.+)
    - source_labels: [__meta_kubernetes_pod_container_name]
      target_label: container
    - source_labels: [__meta_kubernetes_pod_name]
      target_label: pod
    - source_labels: [__meta_kubernetes_namespace]
      target_label: namespace
</code></pre>
