<p>We're running several clusters with AWS's EKS.</p>
<p>Currently all the clusters are already on 1.19 but the NodeGroups are still running on 1.18. The last update of the NodeGroups was in December and there everything works well. The aws-auth ConfigMap wasn't modified since this moment of time.</p>
<p>Now we want to update them. If we either click in the Console on <em>Update</em> or using the following command:</p>
<pre class="lang-sh prettyprint-override"><code>aws eks --region &lt;clusterRegion&gt; update-nodegroup-version --cluster-name=&lt;clusterName&gt; --nodegroup-name=&lt;nodeGroupName&gt;
</code></pre>
<p>...it fails with:</p>
<pre><code>An error occurred (InvalidRequestException) when calling the UpdateNodegroupVersion operation: Nodegroup health has issues other than [ AsgInstanceLaunchFailures, InstanceLimitExceeded, InsufficientFreeAddresses, ClusterUnreachable ]
</code></pre>
<p>A look in the details of the nodegroup shows the following message:</p>
<pre><code>AccessDenied: The aws-auth ConfigMap in your cluster is invalid.
</code></pre>
<p>The related ConfigMap (which works fine for all of us to access it) has the following content (stripped from sensitive information):</p>
<pre><code>mapRoles: |
    - groups:
      - system:bootstrappers
      - system:nodes
      rolearn: arn:aws:iam::&lt;accountId&gt;:role/&lt;ourEksClusterNodeRole&gt;
      username: system:node:{{EC2PrivateDNSName}}
    - groups:
      - system:masters
      rolearn: arn:aws:iam::&lt;accountId&gt;:role/AWSReservedSSO_SystemAdministrator_&lt;someRandomString&gt;
      username: {{SessionName}}
</code></pre>

GreNodge
<p>It turns out that <a href="https://aws.amazon.com/blogs/opensource/integrating-ldap-ad-users-kubernetes-rbac-aws-iam-authenticator-project/" rel="nofollow noreferrer">the proposed way by the AWS documentation to integrate SSO users into the clusters</a> is not compatible with the latest version of EKS.</p>
<p>The placeholder <code>{{SessionName}}</code> cannot be evaluated. So I've had to change it like this:</p>
<pre class="lang-yaml prettyprint-override"><code>mapRoles: |
    - groups:
      - system:bootstrappers
      - system:nodes
      rolearn: arn:aws:iam::&lt;accountId&gt;:role/&lt;ourEksClusterNodeRole&gt;
      username: system:node:{{EC2PrivateDNSName}}
    - groups:
      - system:masters
      rolearn: arn:aws:iam::&lt;accountId&gt;:role/AWSReservedSSO_SystemAdministrator_&lt;someRandomString&gt;
      username: awssso-system-administrator
</code></pre>
<p>The downside of this approach that we lost audit information in the logs.</p>
<p>To get around this (although it is really weird):</p>
<ol>
<li>Adjust the <code>aws-auth</code> ConfigMap like this.</li>
<li>Wait for some minutes.</li>
<li>Trigger the AMI release version upgrade</li>
<li>Wait until it is done.</li>
<li>Change the <code>aws-auth</code> ConfigMap back.</li>
</ol>
