<p>When attempting to drain a node on an AKS K8s cluster using:</p>

<pre><code>kubectl drain ${node_name} --ignore-daemonsets

</code></pre>

<p>I get the following error:</p>

<pre><code>   "The Node \"aks-agentpool-xxxxx-0\" is invalid: []: Forbidden: node updates may only change labels, taints, or capacity (or configSource, if the DynamicKubeletConfig feature gate is enabled)"
</code></pre>

<p>Is there something extra that needs to be done on AKS nodes to allow draining?</p>

<p>(Context: This is part of an automation script I'm writing to drain a kubernetes node for maintenance operations without downtime, so the draining is definitely a prerequisite here)</p>

<p>An additional troubleshooting note:</p>

<p>This command is being run via Ansible's "shell" module, but when the command is run directly in BASH, it works fine.</p>

<p>Further, the ansible is being run via a Jenkins pipeline. Debug statements seem to show:</p>

<ul>
<li>the command being correctly formed and executed.</li>
<li>the context seems correct (so kubeconfig is accessible)</li>
<li>pods can be listed (so kubeconfig is active and correct)</li>
</ul>

Traiano Welcome
<blockquote>
  <p>This command is being run via Ansible's "shell" module, but when the
  command is run directly in BASH, it works fine.</p>
  
  <p>Further, the ansible is being run via a Jenkins pipeline.</p>
</blockquote>

<p>It's good that you added this information because it totally changes the perspective from which we should look at the issue you experience.</p>

<p><strong>For debugging purposes instead of running your command, try to run:</strong></p>

<pre><code>kubectl auth can-i drain node --all-namespaces
</code></pre>

<p>both directly in <code>bash shell</code> as well as via <strong>Ansible's</strong> <code>shell module</code></p>

<p>It should at least give you an answer if this is not a permission issue.</p>

<p>Other commands that you may use to debugging in this case are:</p>

<pre><code>ls -l .kube/config

cat .kube/config

whoami
</code></pre>

<p>Last one to make sure that <strong>Ansible</strong> uses the same <code>user</code>. If you already know that it uses different <code>user</code>, try to run the script as the same <code>user</code> you use for running it in a <strong>bash shell</strong>.</p>

<p>Once you check this, we can continue the debugging process.</p>
