<p>I have deployed the Istio Bookinfo application in a Kubernetes cluster. Following the documentation, I'm trying to measure the average request duration using the following query:</p>

<pre><code>rate(istio_request_duration_milliseconds_sum[1m]) / rate(istio_request_duration_milliseconds_count[1m])
</code></pre>

<p>This query returns me "doubled" results i.e. for each request I get two results, with different <em>Value</em> but same <em>source</em> and <em>destination</em>, one labelled as <em>reporter="source"</em> and the other as <em>reporter="destination"</em>. I was not able to find any clarification about it and I have not clear how these measures work. Why I have two values returned?</p>

<p>The metrics are indeed reported from two sources, once from the request sender and another time from the request receiver. This is a bit redundant, but in some situations the values differ and especially for requests duration: from request sender (<code>reporter=source</code>), the duration stands for the whole request latency (server processing time + network roundtrip) whereas from the request receiver (<code>reporter=destination</code>), it stands mainly for server processing time.</p>

<p>In a few special situations there is no redundancy and metrics are only reported once (be it from source or from destination): e.g. when istio features such as fault injection or traffic mirroring come into play.</p>

<p>As a result, in Prometheus you should always filter all your queries with either <code>{reporter="source"}</code> or <code>{reporter="destination"}</code>.</p>
