<p>I have service A which is a consumer from some queue.</p>
<p>I can monitor and count any consumed message, easily with Prometheus :)</p>
<pre><code>from prometheus_client import start_http_server, Counter

COUNTER_IN_MSGS = Counter('msgs_consumed', 'count consumed messages')
start_http_server(8000)
while(queue not empty):
   A.consume(queue)
   COUNTER_IN_MSGS.inc()
</code></pre>
<p>But than, one day I decide to duplicate my consumer to 10 consumer which do the same {A1, A2..., A10}, by using the same code but running on 10 different dockers (containers on K8s in my case).</p>
<p>How can I monitor them using Prometheus?? Should I change my code and some id to each consumer as label?</p>
<p>What is the best practice to do in order to be able to sum them all together but also count on each by its own?</p>

dev ved
<p>Yes, you should consider using <a href="https://github.com/prometheus/client_python#labels" rel="nofollow noreferrer">labels</a> to disambiguate metrics (e.g. Counters) by instance.</p>
<p>You'll need to determine a unique identifier to use.</p>
<p>Kubernetes provides a <a href="https://kubernetes.io/docs/tasks/inject-data-application/downward-api-volume-expose-pod-information/" rel="nofollow noreferrer">Downward API</a> that enables you to surface information from the Pod to a container. One of these values should be useful.</p>
<p>You can then use PromQL <a href="https://prometheus.io/docs/prometheus/latest/querying/operators/" rel="nofollow noreferrer">ignoring</a> to e.g. sum across Counters and ignore one or more labels.</p>
<p>With this approach, you get to choose whether to sum by instance or across instances.</p>
