<p>I'm trying to change the CPU Manager Policy for a Kubernetes cluster that I manage, as described <a href="https://kubernetes.io/docs/tasks/administer-cluster/cpu-management-policies/" rel="noreferrer">here</a> however, I've run into numerous issues while doing so.</p>

<p>The cluster is running in DigitalOcean and here is what I've tried so far.</p>

<ul>
<li><strong><em>1.</em></strong> Since the article mentions that <code>--cpu-manager-policy</code> is a kubelet option I assume that I cannot change it via the API Server and have to change it manually on each node. (Is this assumption BTW?)</li>
<li><strong><em>2.</em></strong> I <code>ssh</code> into one of the nodes (droplets in DigitalOcean lingo) and run <code>kubelet --cpu-manager-policy=static</code> command as described in the <a href="https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/" rel="noreferrer">kubelet CLI reference here</a>. It gives me the message <code>Flag --cpu-manager-policy has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.</code> </li>
<li><strong><em>3.</em></strong> So I check the file pointed at by the --config flag by running <code>ps aux | grep kubelet</code> and find that its <code>/etc/kubernetes/kubelet.conf</code>.</li>
<li><strong><em>4.</em></strong> I edit the file and add a line <code>cpuManagerPolicy: static</code> to it, and also <code>kubeReserved</code> and <code>systemReserved</code> because they become required fields if specifying <code>cpuManagerPolicy</code>.</li>
<li><strong><em>5.</em></strong> Then I kill the process that was running the process and restart it. A couple other things showed up (delete this file and drain the node etc) that I was able to get through and was able to restart the kubelet ultimately</li>
</ul>

<p>I'm a little lost about the following things</p>

<ul>
<li>How do I need to do this for all nodes? My cluster has 12 of them and doing all of these steps for each seems very inefficient. </li>
<li>Is there any way I can set these params from the globally i.e. cluster-wide rather than doing node by node?</li>
<li>How can I even confirm that what I did actually changed the CPU Manager policy?</li>
</ul>

<p>One issue with Dynamic Configuration is that in case the node fails to restart, the API does not give a reasonable response back that tells you what you did wrong, you'll have to <code>ssh</code> into the node and tail the kubelet logs. Plus, you have to <code>ssh</code> into every node and set the <code>--dynamic-config-dir</code> flag anyways. </p>

<p>The folllowing worked best for me</p>

<ol>
<li>SSH into the node. Edit </li>
</ol>

<pre><code>vim /etc/systemd/system/kubelet.service
</code></pre>

<ol start="2">
<li>Add the following lines</li>
</ol>

<pre><code>  --cpu-manager-policy=static \
  --kube-reserved=cpu=1,memory=2Gi,ephemeral-storage=1Gi \
  --system-reserved=cpu=1,memory=2Gi,ephemeral-storage=1Gi \
</code></pre>

<p>We need to set the <code>--kube-reserved</code> and <code>--system-reserved</code> flags because they're prerequisties to setting the <code>--cpu-manager-policy</code> flag</p>

<ol start="3">
<li>Then drain the node and delete the following folder </li>
</ol>

<pre><code>rm -rf /var/lib/kubelet/cpu_manager_state
</code></pre>

<ol start="4">
<li>Restart the kubelet </li>
</ol>

<pre><code>sudo systemctl daemon-reload
sudo systemctl stop kubelet
sudo systemctl start kubelet
</code></pre>

<ol start="6">
<li>uncordon the node and check the policy. This assumes that you're running <code>kubectl proxy</code> on port 8001.</li>
</ol>

<pre><code>curl -sSL "http://localhost:8001/api/v1/nodes/${NODE_NAME}/proxy/configz" | grep cpuManager
</code></pre>
