<p>I noticed some of my clusters were reporting a CPUThrottlingHigh alert for metrics-server-nanny container (image: gke.gcr.io/addon-resizer:1.8.11-gke.0) in GKE.  I couldn't see a way to configure this container to give it more CPU because it's automatically deployed as part of the metrics-server pod, and Google automatically resets any changes to the deployment/pod resource settings.</p>
<p>So out of curiosity, I created a small kubernetes cluster in GKE (3 standard nodes) with autoscaling turned on to scale up to 5 nodes.  No apps or anything installed.  Then I installed the kube-prometheus monitoring stack (<a href="https://github.com/prometheus-operator/kube-prometheus" rel="nofollow noreferrer">https://github.com/prometheus-operator/kube-prometheus</a>) which includes the CPUThrottlingHigh alert.  Soon after installing the monitoring stack, this same alert popped up for this container.  I don't see anything in the logs of this container or the related metrics-server-nanny container.</p>
<p>Also, I don't notice this same issue on AWS or Azure because while they do have a similar metrics-server pod in the kube-system namespace, they do not contain the sidecar metrics-server-nanny container in the pod.</p>
<p>Has anyone seen this or something similar?  Is there a way to give this thing more resources without Google overwriting config changes?</p>

pgier
<p>This is a known issue in <a href="https://github.com/kubernetes/kubernetes/issues/67577" rel="nofollow noreferrer">Kubernetes</a> that CFS leads to Throttling Pods that exhibit a spikey CPU usage pattern. As Kubernetes / GKE uses to implement CPU quotas, this is causing pods to get throttled even when they really aren't busy.</p>
<p>Kubernetes uses CFS quotas to enforce CPU limits for the pods running an application. The Completely Fair Scheduler (CFS) is a process scheduler that handles CPU resource allocation for executing processes, based on time period and not on available CPU power.</p>
<p>We have no direct control over CFS via Kubernetes, so the only solution is to disable it.  This is done via node config.</p>
<p>Allow users to tune Kubelet configs &quot;<strong>CPUManagerPolicy</strong>&quot; and &quot;<strong>CPUCFSQuota</strong>”</p>
<p>The workaround is to temporarily disable Kubernetes CFS quotas entirely (kubelet's flag <strong>--cpu-cfs-quota=false</strong>)</p>
<pre><code>    $ cat node-config.yaml
    kubeletConfig:
    cpuCFSQuota: false
    cpuManagerPolicy: static

    $ gcloud container clusters create --node-config=node-config.yaml
</code></pre>
<p>gcloud will map the fields from the YAML node config file to the newly added GKE API fields.</p>
