<p>I'm setting up a CI/CD pipeline for deploying a Kubernetes-based application in an automated way. Part of this deployment involves creating other service accounts and their associated roles.</p>
<p>When my pipeline runs, deployment fails with this error message:</p>
<pre><code>Error: roles.rbac.authorization.k8s.io &quot;mongodb-kubernetes-operator&quot; is forbidden: user &quot;cicd-bot@my-project.iam.gserviceaccount.com&quot; (groups=[&quot;system:authenticated&quot;]) is attempting to grant RBAC permissions not currently held:
│ {APIGroups:[&quot;&quot;], Resources:[&quot;configmaps&quot;], Verbs:[&quot;list&quot; &quot;watch&quot; &quot;create&quot; &quot;update&quot; &quot;patch&quot; &quot;get&quot; &quot;delete&quot;]}
│ {APIGroups:[&quot;&quot;], Resources:[&quot;pods&quot;], Verbs:[&quot;list&quot; &quot;watch&quot; &quot;create&quot; &quot;update&quot; &quot;patch&quot; &quot;get&quot; &quot;delete&quot;]}
│ {APIGroups:[&quot;&quot;], Resources:[&quot;secrets&quot;], Verbs:[&quot;list&quot; &quot;watch&quot; &quot;create&quot; &quot;update&quot; &quot;patch&quot; &quot;get&quot; &quot;delete&quot;]}
│ {APIGroups:[&quot;&quot;], Resources:[&quot;services&quot;], Verbs:[&quot;list&quot; &quot;watch&quot; &quot;create&quot; &quot;update&quot; &quot;patch&quot; &quot;get&quot; &quot;delete&quot;]}
│ {APIGroups:[&quot;apps&quot;], Resources:[&quot;statefulsets&quot;], Verbs:[&quot;list&quot; &quot;watch&quot; &quot;create&quot; &quot;update&quot; &quot;patch&quot; &quot;get&quot; &quot;delete&quot;]}
│ {APIGroups:[&quot;mongodbcommunity.mongodb.com&quot;], Resources:[&quot;mongodbcommunity&quot;], Verbs:[&quot;list&quot; &quot;watch&quot; &quot;update&quot; &quot;patch&quot; &quot;get&quot;]}
│ {APIGroups:[&quot;mongodbcommunity.mongodb.com&quot;], Resources:[&quot;mongodbcommunity/finalizers&quot;], Verbs:[&quot;list&quot; &quot;watch&quot; &quot;update&quot; &quot;patch&quot; &quot;get&quot;]}
│ {APIGroups:[&quot;mongodbcommunity.mongodb.com&quot;], Resources:[&quot;mongodbcommunity/spec&quot;], Verbs:[&quot;list&quot; &quot;watch&quot; &quot;update&quot; &quot;patch&quot; &quot;get&quot;]}
│ {APIGroups:[&quot;mongodbcommunity.mongodb.com&quot;], Resources:[&quot;mongodbcommunity/status&quot;], Verbs:[&quot;list&quot; &quot;watch&quot; &quot;update&quot; &quot;patch&quot; &quot;get&quot;]}
│ 
│   with module.db_document.kubernetes_role.operator_mongodb,
│   on modules/db_document/main.tf line 17, in resource &quot;kubernetes_role&quot; &quot;operator_mongodb&quot;:
│   17: resource &quot;kubernetes_role&quot; &quot;operator_mongodb&quot; {
│ 
</code></pre>
<p>The error seemed straightforward enough: my service account can't grant permissions that it does not have. Since the error message mentioned my GCP service account, <code>cicd-bot@my-project.iam.gserviceaccount.com</code>, I added to my role definition what I believed to be the matching permissions.</p>
<p>Below is my resulting role. It has create, delete, get, list, and update permissions for configMaps, pods, secrets, services, statefulsets , and thirdPartyObjects, which I believed should cover the requirements.</p>
<pre><code>resource &quot;google_project_iam_custom_role&quot; &quot;cicd_bot_role&quot; {
  project = var.project
  role_id = &quot;cicd_bot&quot;
  title   = &quot;CICD Bot&quot;
  permissions = [
    &quot;artifactregistry.repositories.downloadArtifacts&quot;,
    &quot;artifactregistry.repositories.uploadArtifacts&quot;,
    &quot;compute.instanceGroupManagers.get&quot;,
    &quot;container.clusters.get&quot;,
    &quot;container.configMaps.create&quot;,
    &quot;container.configMaps.delete&quot;,
    &quot;container.configMaps.get&quot;,
    &quot;container.configMaps.list&quot;,
    &quot;container.configMaps.update&quot;,
    &quot;container.cronJobs.create&quot;,
    &quot;container.cronJobs.delete&quot;,
    &quot;container.cronJobs.get&quot;,
    &quot;container.cronJobs.update&quot;,
    &quot;container.customResourceDefinitions.create&quot;,
    &quot;container.customResourceDefinitions.delete&quot;,
    &quot;container.customResourceDefinitions.get&quot;,
    &quot;container.customResourceDefinitions.list&quot;,
    &quot;container.customResourceDefinitions.update&quot;,
    &quot;container.deployments.create&quot;,
    &quot;container.deployments.delete&quot;,
    &quot;container.deployments.get&quot;,
    &quot;container.deployments.update&quot;,
    &quot;container.ingresses.create&quot;,
    &quot;container.ingresses.delete&quot;,
    &quot;container.ingresses.get&quot;,
    &quot;container.ingresses.update&quot;,
    &quot;container.jobs.create&quot;,
    &quot;container.jobs.delete&quot;,
    &quot;container.jobs.get&quot;,
    &quot;container.jobs.update&quot;,
    &quot;container.namespaces.get&quot;,
    &quot;container.persistentVolumeClaims.create&quot;,
    &quot;container.persistentVolumeClaims.delete&quot;,
    &quot;container.persistentVolumeClaims.get&quot;,
    &quot;container.persistentVolumeClaims.update&quot;,
    &quot;container.pods.create&quot;,
    &quot;container.pods.delete&quot;,
    &quot;container.pods.get&quot;,
    &quot;container.pods.list&quot;,
    &quot;container.pods.update&quot;,
    &quot;container.roleBindings.create&quot;,
    &quot;container.roleBindings.delete&quot;,
    &quot;container.roleBindings.get&quot;,
    &quot;container.roleBindings.update&quot;,
    &quot;container.roles.create&quot;,
    &quot;container.roles.delete&quot;,
    &quot;container.roles.get&quot;,
    &quot;container.roles.update&quot;,
    &quot;container.secrets.create&quot;,
    &quot;container.secrets.delete&quot;,
    &quot;container.secrets.get&quot;,
    &quot;container.secrets.list&quot;,
    &quot;container.secrets.update&quot;,
    &quot;container.serviceAccounts.create&quot;,
    &quot;container.serviceAccounts.delete&quot;,
    &quot;container.serviceAccounts.get&quot;,
    &quot;container.serviceAccounts.update&quot;,
    &quot;container.services.create&quot;,
    &quot;container.services.delete&quot;,
    &quot;container.services.get&quot;,
    &quot;container.services.list&quot;,
    &quot;container.services.update&quot;,
    &quot;container.statefulSets.create&quot;,
    &quot;container.statefulSets.delete&quot;,
    &quot;container.statefulSets.get&quot;,
    &quot;container.statefulSets.list&quot;,
    &quot;container.statefulSets.update&quot;,
    &quot;container.thirdPartyObjects.create&quot;,
    &quot;container.thirdPartyObjects.delete&quot;,
    &quot;container.thirdPartyObjects.get&quot;,
    &quot;container.thirdPartyObjects.list&quot;,
    &quot;container.thirdPartyObjects.update&quot;,
    &quot;dns.changes.create&quot;,
    &quot;dns.changes.get&quot;,
    &quot;dns.resourceRecordSets.get&quot;,
    &quot;dns.resourceRecordSets.list&quot;,
    &quot;dns.resourceRecordSets.update&quot;,
    &quot;storage.buckets.get&quot;,
    &quot;storage.objects.create&quot;,
    &quot;storage.objects.delete&quot;,
    &quot;storage.objects.get&quot;,
    &quot;storage.objects.list&quot;,
  ]
}
</code></pre>
<p>However, after deploying this, the error remained the same. I wondered if adding equivalent permissions on the kubernetes side was necessary, so I created the following ClusterRole and ClusterRoleBinding, too.</p>
<pre><code>resource &quot;kubernetes_cluster_role&quot; &quot;cicd_bot&quot; {
  metadata {
    name = kubernetes_service_account.cicd_bot.metadata[0].name
  }
  rule {
    api_groups = [&quot;&quot;]
    resources  = [&quot;namespaces&quot;]
    verbs      = [&quot;create&quot;, &quot;delete&quot;, &quot;get&quot;]
  }
  rule {
    api_groups = [&quot;&quot;]
    resources  = [&quot;configmaps&quot;]
    verbs      = [&quot;list&quot;, &quot;watch&quot;, &quot;create&quot;, &quot;update&quot;, &quot;patch&quot;, &quot;get&quot;, &quot;delete&quot;]
  }
  rule {
    api_groups = [&quot;&quot;]
    resources  = [&quot;pods&quot;]
    verbs      = [&quot;list&quot;, &quot;watch&quot;, &quot;create&quot;, &quot;update&quot;, &quot;patch&quot;, &quot;get&quot;, &quot;delete&quot;]
  }
  rule {
    api_groups = [&quot;&quot;]
    resources  = [&quot;secrets&quot;]
    verbs      = [&quot;list&quot;, &quot;watch&quot;, &quot;create&quot;, &quot;update&quot;, &quot;patch&quot;, &quot;get&quot;, &quot;delete&quot;]
  }
  rule {
    api_groups = [&quot;&quot;]
    resources  = [&quot;services&quot;]
    verbs      = [&quot;list&quot;, &quot;watch&quot;, &quot;create&quot;, &quot;update&quot;, &quot;patch&quot;, &quot;get&quot;, &quot;delete&quot;]
  }
  rule {
    api_groups = [&quot;apps&quot;]
    resources  = [&quot;statefulsets&quot;]
    verbs      = [&quot;list&quot;, &quot;watch&quot;, &quot;create&quot;, &quot;update&quot;, &quot;patch&quot;, &quot;get&quot;, &quot;delete&quot;]
  }
  rule {
    api_groups = [&quot;mongodbcommunity.mongodb.com&quot;]
    resources  = [&quot;mongodbcommunity&quot;]
    verbs      = [&quot;list&quot;, &quot;watch&quot;, &quot;update&quot;, &quot;patch&quot;, &quot;get&quot;]
  }
  rule {
    api_groups = [&quot;mongodbcommunity.mongodb.com&quot;]
    resources  = [&quot;mongodbcommunity/finalizers&quot;]
    verbs      = [&quot;list&quot;, &quot;watch&quot;, &quot;update&quot;, &quot;patch&quot;, &quot;get&quot;]
  }
  rule {
    api_groups = [&quot;mongodbcommunity.mongodb.com&quot;]
    resources  = [&quot;mongodbcommunity/spec&quot;]
    verbs      = [&quot;list&quot;, &quot;watch&quot;, &quot;update&quot;, &quot;patch&quot;, &quot;get&quot;]
  }
  rule {
    api_groups = [&quot;mongodbcommunity.mongodb.com&quot;]
    resources  = [&quot;mongodbcommunity/status&quot;]
    verbs      = [&quot;list&quot;, &quot;watch&quot;, &quot;update&quot;, &quot;patch&quot;, &quot;get&quot;]
  }
}

resource &quot;kubernetes_cluster_role_binding&quot; &quot;cicd_bot&quot; {
  metadata {
    name = kubernetes_service_account.cicd_bot.metadata[0].name
  }
  subject {
    kind      = &quot;ServiceAccount&quot;
    namespace = kubernetes_service_account.cicd_bot.metadata[0].namespace
    name      = kubernetes_service_account.cicd_bot.metadata[0].name
  }
  role_ref {
    api_group = &quot;rbac.authorization.k8s.io&quot;
    kind      = &quot;ClusterRole&quot;
    name      = kubernetes_cluster_role.cicd_bot.metadata[0].name
  }
}
</code></pre>
<p>Unfortunately, the pipeline still fails with the same error. I've been able to overcome a similar error in the past, but not this time. What am I missing?</p>
<p>UPDATE: I was able to deploy successfully by attaching the role <code>roles/container.admin</code> to my service account. So now I need to figure out which permission <code>roles/container.admin</code> has that my custom role does not.</p>

<p>Sadly, the one permission that was missing was</p>
<pre><code>container.roles.escalate
</code></pre>
<p>Even including every other <code>container.*</code> permission was insufficient; <code>container.roles.escalate</code> was still needed.</p>
<p>This is unfortunate because it makes the cluster more vulnerable to permissions escalation attacks. If there is a safer way to accomplish this, I'd love to hear it. I won't mark my own answer as &quot;correct&quot; because I'm not satisfied with it. But hey, at least it's working...</p>
