<p>For example, kubelet(cAdvisor) <code>container_cpu_usage_seconds_total</code> has value with some parameter (e.g. pod, namespace).</p>

<p>I wonder how to summarize this kind of values into Service(for example, CPU usage per service)? I understand that Service is a set of pods so that just aggregating these values per pod to service, but I do not know how?</p>

<p>Is there any aggregation method to Service?  Or, <code>process_cpu_seconds_total</code> is a kind of aggregated value per service of 'container_cpu_usage_seconds_total'?</p>

<p>Thank you for your help!</p>

<p>In general, cAdvisor collects metrics about containers and doesn't know anything about Services. If you want to aggregate by Service, you need to manually select the metrics of the Pod that belong to this Service.</p>

<p>For example, if your cAdvisor metrics are in Prometheus, you can use this PromQL query:</p>

<pre><code>sum(rate(container_cpu_usage_seconds_total{pod_name=~"myprefix-*"}[2m]))
</code></pre>

<p>This adds up the CPU usages of all containers of all Pods that have a name starting with <code>myprefix-</code>.</p>

<p>Or if you have the <a href="https://github.com/kubernetes/community/blob/master/contributors/design-proposals/instrumentation/resource-metrics-api.md" rel="nofollow noreferrer">Resource Metrics API</a> enabled (i.e. the <a href="https://github.com/kubernetes-incubator/metrics-server" rel="nofollow noreferrer">Metrics Server</a> installed), you can query the CPU usage of a specific Pod (in fractions of a CPU core) with:</p>

<pre><code>kubectl get --raw="/apis/metrics.k8s.io/v1beta1/namespaces/{namespace}/pods/{pod}"
</code></pre>

<p>To get the total usage of a Service, you would need to iterate through all the Pods of the Service, extract the values, and add them together.</p>

<p>In general, Service is a Kubernetes concept and does not exist in cAdvisor, which is an independent project and just happens to be used in Kubernetes.</p>
