<p>Prometheus has metrics such as <code>container_cpu_usage_seconds_total</code>. However, they are only grouped by pod. How can I group them by deployment/cronjobs/etc?</p>

<p>I was able to handle this with the following query:</p>
<p><code>((label_replace((rate(container_cpu_usage_seconds_total{image!=&quot;&quot;}[2m]) * on(pod) group_left(owner_name) (sum without (instance) (kube_pod_owner))), &quot;replicaset&quot;, &quot;$1&quot;, &quot;owner_name&quot;, &quot;(.*)&quot;)) * on(replicaset) group_left(owner_name) (sum without (instance) (kube_replicaset_owner{})))</code></p>
<p>Here is the explanation:</p>
<ul>
<li>Join <code>container_cpu_usage_seconds_total</code> with <code>kube_pod_owner</code> on <code>pod</code></li>
<li>Copy over the <code>owner_name</code> from <code>kube_pod_owner</code></li>
<li>Use <code>label_replace</code> to rename <code>kube_pod_owner</code>'s <code>owner_name</code> to <code>replicaset</code></li>
<li>Join that with kube_replicaset_owner on <code>replicaset</code></li>
<li>Copy over the <code>owner_name</code> from <code>kube_replicaset_owner</code> (this value is your deployment etc)</li>
</ul>
<p>The <code>without (instance)</code> are used to remove the <code>instance</code> field from the joined sets. Because there can be multiple instances for a single deployment, this can cause issues.</p>
<p>Lastly, the <code>rate</code> function is called on <code>container_cpu_usage_seconds_total</code> directly at the innermost area because otherwise Prometheus complains about <code>parse error: ranges only allowed for vector selectors</code>. Placing it in the innermost area is a workaround.</p>
