<p>I'm looking for a Prometheus metric that would allow me to monitor the time pods spend in the <code>terminating</code> state before vanishing into the void.  </p>

<p>I've tried playing around with <code>kube_pod_container_status_terminated</code> but it only seems to register pods once they finish the termination process, but don't help me understand how long does it take to terminate a pod.<br>
I've also looked at <code>kube_pod_status_phase</code> which I found out about in this channel a while ago but it also seems to lack this insight.</p>

<p>I'm currently collecting metrics on my k8s workload using cAdvisor, kube-state-metrics and the prometheus node-exporter, but would happily considering additional collectors if they contain the desired data.<br>
A non-prometheus solution would also be great.<br>
Any ideas? Thanks!</p>

<p>Kubernetes itself, Heapster and metrics-server don't provide such metrics, but you can get metrics close to what you've mentioned by installing <a href="https://github.com/kubernetes/kube-state-metrics" rel="nofollow noreferrer">kube-state-metrics</a>. It has several pod metrics that reflect pods state:</p>

<pre><code>kube_pod_status_phase
kube_pod_container_status_terminated
kube_pod_container_status_terminated_reason
kube_pod_container_status_last_terminated_reason
</code></pre>

<p>You can find the full list of pods metrics, provided by <code>kube-state-metrics</code> in <a href="https://github.com/kubernetes/kube-state-metrics/blob/master/Documentation/pod-metrics.md" rel="nofollow noreferrer">Documentation</a>.</p>

<p>There is also <a href="https://hub.docker.com/r/bitnami/kube-state-metrics/" rel="nofollow noreferrer">Bitnami Helm chart</a> that could simplify the installation of <code>kube-state-metrics</code>.</p>
