<p>I am deploying Prometheus with Thanos with the given config:</p>

<pre><code>global:
  scrape_interval: 15s  # By default, scrape targets every 15 seconds.

 # Attach these labels to any time series or alerts when communicating with
 # external systems (federation, remote storage, Alertmanager).
 external_labels:
   monitor: %%CLUSTER%%
   replica: $(HOSTNAME)
</code></pre>

<p>I got this from the Thanos example documents but I am not able to make this work.
"%%CLUSTER%% gives "cannot start with the given token" error whereas the HOSTNAME substitution never happens.</p>

<p>I read on this issue that variable substitution is not supported but the Thanos docs says otherwise.
<a href="https://github.com/prometheus/prometheus/issues/2357" rel="nofollow noreferrer">https://github.com/prometheus/prometheus/issues/2357</a></p>

<p>Is there any other way to implement this?</p>

<p>There is a standard way to do it since Prometheus 2.27.</p>
<p>See <a href="https://github.com/prometheus/prometheus/blob/main/docs/disabled_features.md#expand-environment-variables-in-external-labels" rel="nofollow noreferrer">disabled_features</a></p>
<p><code>--enable-feature=expand-external-labels</code></p>
<p>Replace <code>${var}</code> or <code>$var</code> in the external_labels values according to the values of the current environment variables. References to undefined variables are replaced by the empty string.</p>
<p>And this <a href="https://github.com/prometheus/prometheus/issues/6047#issuecomment-810874415" rel="nofollow noreferrer">reply</a> by <a href="https://github.com/roidelapluie" rel="nofollow noreferrer">roidelaplui</a></p>
