<p>I have setup statsd-exporter to scrape metric from gunicorn web server. My goal is to filter request duration metric only for successful request(non 5xx), however in statsd-exporter there is no way to tag status code in duration metric. Can anyone suggest a way to add status code in request duration metric or a way to filter only successful request duration in prometheus.</p>

<p>In particular I want to extract successful request duration hitogram from statsd-exporter to prometheus. </p>

Santosh Kumar
<p>To export successful request duration histogram metrics from gunicorn web server to prometheus you would need to add this functionality in gunicorn sorcecode.</p>

<p>First take a look at the code that exports statsd metrics <a href="https://github.com/benoitc/gunicorn/blob/aa8b258f937867a8a453b426e5c26db84a8ab879/gunicorn/instrument/statsd.py#L99" rel="nofollow noreferrer">here</a>.
You should see this peace of code:</p>

<pre><code>status = resp.status
...
self.histogram("gunicorn.request.duration", duration_in_ms)
</code></pre>

<p>By changing the code to sth like this:</p>

<pre><code>self.histogram("gunicorn.request.duration.%d" % status, duration_in_ms)
</code></pre>

<p>from this moment you will have metrics names exported with status codes like <code>gunicorn_request_duration_200</code> or <code>gunicorn_request_duration_404</code> etc.</p>

<p>You can also modify it a little bit and move status codes to label by adding a configuration like below to your <code>statsd_exporter</code>:</p>

<pre><code>mappings:
  - match: gunicorn.request.duration.*
    name: "gunicorn_http_request_duration"
    labels:
      status: "$1"
      job: "gunicorn_request_duration"
</code></pre>

<p>So your metrics will now look like this:</p>

<pre><code># HELP gunicorn_http_request_duration Metric autogenerated by statsd_exporter.
# TYPE gunicorn_http_request_duration summary
gunicorn_http_request_duration{job="gunicorn_request_duration",status="200",quantile="0.5"} 2.4610000000000002e-06
gunicorn_http_request_duration{job="gunicorn_request_duration",status="200",quantile="0.9"} 2.4610000000000002e-06
gunicorn_http_request_duration{job="gunicorn_request_duration",status="200",quantile="0.99"} 2.4610000000000002e-06
gunicorn_http_request_duration_sum{job="gunicorn_request_duration",status="200"} 2.4610000000000002e-06
gunicorn_http_request_duration_count{job="gunicorn_request_duration",status="200"} 1
gunicorn_http_request_duration{job="gunicorn_request_duration",status="404",quantile="0.5"} 3.056e-06
gunicorn_http_request_duration{job="gunicorn_request_duration",status="404",quantile="0.9"} 3.056e-06
gunicorn_http_request_duration{job="gunicorn_request_duration",status="404",quantile="0.99"} 3.056e-06
gunicorn_http_request_duration_sum{job="gunicorn_request_duration",status="404"} 3.056e-06
gunicorn_http_request_duration_count{job="gunicorn_request_duration",status="404"} 1
</code></pre>

<p>And now to query all metrics except these with 5xx status in prometheus you can run:</p>

<pre><code>gunicorn_http_request_duration{status=~"[^5].*"}
</code></pre>

<p>Let me know if it was helpful.</p>
