<p>I am running different versions of our application in different namespaces and I have set up a prometheus and grafana stack to monitor them. I am using below promql for getting the cpu usage of different pods (as percentage of 1 core) and the value that it returns is matching the values that I get from the <code>kubectl top pods -n namespace</code>:</p>

<pre><code>sum (rate (container_cpu_usage_seconds_total{id!="/",namespace=~"$Namespace",pod=~"^$Deployment.*$"}[1m])) by (pod)*100
</code></pre>

<p>The problem is I want to get the total cpu usage of all pods in a namespace cluster-wide and I tried different queries but the values that they return is not matching the total cpu usage that I get from the above promql or <code>kubectl top pods -n namespace</code>.</p>

<p>The promql queries that I tried:</p>

<pre><code>sum (rate (container_cpu_usage_seconds_total{namespace=~"$Namespace",pod=~"^$Deployment.*$"}[1m])) by (namespace)
sum (rate (container_cpu_usage_seconds_total{namespace=~"$Namespace",pod=~"^$Deployment.*$"}[1m]))
</code></pre>

<p>I am using the <code>Singlestat</code> for this and also at <code>visualization</code> from <code>Value</code> section I tried different <code>show</code> methods such as Average, total, current but non returned the correct value.</p>

<p>My question is how I can get the total cpu usage of all the pods in a namespace cluster-wide?</p>

AVarf
<p>The following PromQL query should return summary per-namespace CPU usage across all the pods in Kuberentes (the CPU usage is expressed in the number of used CPU cores):</p>
<pre><code>sum(rate(container_cpu_usage_seconds_total{container!=&quot;&quot;}[5m])) by (namespace)
</code></pre>
<p>The <code>{container!~&quot;&quot;}</code> filter is needed for removing CPU usage metrics for <code>cgroups hierarchy</code>, since these metrics are already included with non-empty <code>container</code> labels.</p>
<p>The query uses the following PromQL functions:</p>
<ul>
<li><a href="https://docs.victoriametrics.com/MetricsQL.html#sum" rel="nofollow noreferrer">sum()</a></li>
<li><a href="https://docs.victoriametrics.com/MetricsQL.html#rate" rel="nofollow noreferrer">rate()</a></li>
</ul>
