<p>I'm using <a href="https://github.com/prometheus-community/helm-charts/tree/main/charts/prometheus" rel="nofollow noreferrer">prometheus-community/prometheus</a> chart</p>
<p>I'd like to add the following labels automatically to any alert manager rule firing</p>
<ul>
<li>env=<strong>prod</strong></li>
<li>cluster=<strong>project-prod-eks</strong></li>
</ul>
<p>so that I don't these labels manually to each alert rule.</p>
<pre class="lang-yaml prettyprint-override"><code> - alert: NGINXTooMany400s
   expr: 100 * ( sum( nginx_ingress_controller_requests{status=~&quot;4.+&quot;} ) / sum(nginx_ingress_controller_requests) ) &gt; 5
   for: 1m
   labels:
     severity: warning
     env: prod
     cluster: project-prod-eks              &lt;---------------HOW to inject them?
   annotations:
     description: Too many 4XXs
     summary: More than 5% of all requests returned 4XX, this requires your attention
</code></pre>
<p>so that I can do something like</p>
<pre><code> - alert: NGINXTooMany400s
   expr: 100 * ( sum( nginx_ingress_controller_requests{status=~&quot;4.+&quot;} ) / sum(nginx_ingress_controller_requests) ) &gt; 5
   for: 1m
   labels:
     severity: warning
   annotations:
     description: Too many 4XXs on {{ $labels.env }} / {{ $labels.cluster }}  &lt;----- THIS
     summary: More than 5% of all requests returned 4XX, this requires your attention
</code></pre>
<h3>Any ideas?</h3>

DmitrySemenov
<p>You can add <code>external_labels</code> to your <a href="https://prometheus.io/docs/prometheus/latest/configuration/configuration/" rel="nofollow noreferrer">prometheus.yml</a>:</p>
<pre class="lang-yaml prettyprint-override"><code>global:
  # The labels to add to any time series or alerts when communicating with
  # external systems (federation, remote storage, Alertmanager).
  external_labels:
    env: prod
    cluster: project-prod-eks
</code></pre>
<p>The community chart has it in <a href="https://github.com/prometheus-community/helm-charts/blob/main/charts/prometheus/values.yaml#L1345" rel="nofollow noreferrer">values.yml</a>:</p>
<pre class="lang-yaml prettyprint-override"><code>serverFiles:
  prometheus.yml:
    global:
      external_labels:
        foo: bar
    ...
</code></pre>
