<p>I understand with Prometheus we can set up alerting rules which can detect and raise an alert if a pod crashes. </p>

<p>I want to understand how does Prometheus itself know when a pod crashed or is stuck in pending state.</p>

<ul>
<li>Does it know this when it is trying to scrape metrics from pod's http endpoint port?</li>
</ul>

<p>OR</p>

<ul>
<li>Does Prometheus get the pod status information from Kubernetes?</li>
</ul>

<p>The reason why I'm asking this is because I want to set up Prometheus to monitor existing pods that I have already deployed. I want to be alerted if a pod keeps crashing or if it is stuck in pending state. And I want to know if Prometheus can detect these alerts without making any modifications to the code inside the existing pods.</p>

<p>use <code>sum(kube_pod_container_status_waiting_reason) by (reason)</code> to get all the container waiting reasons if any</p>
