<p>I am deploying in Azure AKS a regular deployment and i want to use keyvault to store my secrets to get access to a database.</p>
<p>This is my deployment file:</p>
<pre><code>apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: sonarqube
  name: sonarqube
spec:
  selector:
    matchLabels:
      app: sonarqube
  replicas: 1
  template:
    metadata:
      labels:
        app: sonarqube
    spec:
      containers:
        - name: sonarqube
          image: sonarqube:8.9-developer
          resources:
            requests:
              cpu: 500m
              memory: 1024Mi
            limits:
              cpu: 2000m
              memory: 4096Mi
          volumeMounts:
          - mountPath: &quot;/mnt/secrets/&quot;
            name: secrets-store-inline
          - mountPath: &quot;/opt/sonarqube/data/&quot;
            name: sonar-data-new
          - mountPath: &quot;/opt/sonarqube/extensions/plugins/&quot;
            name: sonar-extensions-new2
          env:
          - name: &quot;SONARQUBE_JDBC_USERNAME&quot;
            valueFrom:
              secretKeyRef:
                name: test-secret
                key: username
          - name: &quot;SONARQUBE_JDBC_PASSWORD&quot;
            valueFrom:
              secretKeyRef:
                name: test-secret
                key: password
          - name: &quot;SONARQUBE_JDBC_URL&quot;
            valueFrom:
              configMapKeyRef:
                name: sonar-config
                key: url
          ports:
          - containerPort: 9000
            protocol: TCP
      volumes:
      - name: sonar-data-new
        persistentVolumeClaim:
          claimName: sonar-data-new
      - name: sonar-extensions-new2
        persistentVolumeClaim:
          claimName: sonar-extensions-new2
      - name: secrets-store-inline
        csi:
          driver: secrets-store.csi.k8s.io
          readOnly: true
          volumeAttributes:
           secretProviderClass: &quot;azure-kv-provider&quot;
</code></pre>
<p>and this is my secret storage class:</p>
<pre><code>kind: SecretProviderClass
metadata:
  name: azure-kv-provider
spec:
  provider: azure
  secretObjects:
   - data:
      - key: username
        objectName: username
      - key: password
        objectName: password
     secretName: test-secret
     type: Opaque
  parameters:
    usePodIdentity: &quot;false&quot;  
    useAssignedIdentity: &quot;true&quot;
    userAssignedIdentityID: &quot;zzzz-zzzz-zzzz-zzzz-zzzz&quot;
    keyvaultName: &quot;dbkvtz&quot; 
    cloudName: &quot;&quot;   
    objects:  |
      array:
        - |
          objectName: test
          objectType: secret
          objectAlias: username
          objectVersion: &quot;&quot; 
        - |
          objectName: test
          objectType: secret 
          objectAlias: password  
          objectVersion: &quot;&quot;       
    resourceGroup: &quot;myresourcegroup&quot;  
    subscriptionId: &quot;yyyy-yyyy-yyyy-yyy-yyyy&quot;       
    tenantId: &quot;xxxx-xxxx-xxxx-xxx-xxxx&quot; 

</code></pre>
<p>Where &quot;zzzz-zzzz-zzzz-zzzz-zzzz&quot; is the Client ID of the created Managed Identity.</p>
<p>In the Key Vault that i created &quot;dbkvtz&quot; i added through &quot;Access Policy&quot; the Managed Identity that i created. On the other hand in &quot;Manage Identity&quot; i am not able to add any role in &quot;Azure Role Assignement&quot; -- No role assignments found for the selected subscription. I don't know if it is necessary to add any role there.</p>
<p>The AKS cluster is setup for system assigned managed identity. I want to use Managed Identities to get access to the key vaults so i created a managed identity with client id &quot;zzzz-zzzz-zzzz-zzzz-zzzz&quot; (where is &quot;z&quot; a value from 0-9a-z).</p>
<p>I am not too familiar with keyvault integration in AKS so i am not sure if the config is ok.</p>
<p>I am getting this error:</p>
<p><strong>kubectl describe pods:</strong></p>
<pre><code>  Normal   Scheduled    19m                  default-scheduler  Successfully assigned default/sonarqube-6bdb9cfc85-npbfw to aks-agentpool-16966606-vmss000000
  Warning  FailedMount  5m43s (x5 over 16m)  kubelet            Unable to attach or mount volumes: unmounted volumes=[secrets-store-inline], unattached volumes=[secrets-store-inline sonar-data-new sonar-extensions-new2 default-token-t45tw]: timed out waiting for the condition
  Warning  FailedMount  3m27s                kubelet            Unable to attach or mount volumes: unmounted volumes=[secrets-store-inline], unattached volumes=[default-token-t45tw secrets-store-inline sonar-data-new sonar-extensions-new2]: timed out waiting for the condition
  Warning  FailedMount  71s (x2 over 10m)    kubelet            Unable to attach or mount volumes: unmounted volumes=[secrets-store-inline], unattached volumes=[sonar-data-new sonar-extensions-new2 default-token-t45tw secrets-store-inline]: timed out waiting for the condition
  Warning  FailedMount  37s (x17 over 19m)   kubelet            MountVolume.SetUp failed for volume &quot;secrets-store-inline&quot; : rpc error: code = Unknown desc = failed to mount secrets store objects for pod default/sonarqube-6bdb9cfc85-npbfw, err: rpc error: code = Unknown desc = failed to mount objects, error: failed to create auth config, error: failed to get credentials, nodePublishSecretRef secret is not set

</code></pre>
<p>logs az aks show -g RG -n SonarQubeCluster</p>
<pre><code>{
  &quot;aadProfile&quot;: null,
  &quot;addonProfiles&quot;: {
    &quot;azurepolicy&quot;: {
      &quot;config&quot;: null,
      &quot;enabled&quot;: true,
      &quot;identity&quot;: {
        &quot;clientId&quot;: &quot;yy&quot;,
        &quot;objectId&quot;: &quot;zz&quot;,
        &quot;resourceId&quot;: &quot;/subscriptions/xx/resourcegroups/MC_xx_SonarQubeCluster_southcentralus/providers/Microsoft.ManagedIdentity/userAssignedIdentities/azurepolicy-sonarqubecluster&quot;
      }
    },
    &quot;httpApplicationRouting&quot;: {
      &quot;config&quot;: null,
      &quot;enabled&quot;: false,
      &quot;identity&quot;: null
    },
    &quot;omsagent&quot;: {
      &quot;config&quot;: {
        &quot;logAnalyticsWorkspaceResourceID&quot;: &quot;/subscriptions/xx/resourceGroups/DefaultResourceGroup-SCUS/providers/Microsoft.OperationalInsights/workspaces/DefaultWorkspace-44e26024-4977-4419-8d23-0e1e22e8804e-SCUS&quot;
      },
      &quot;enabled&quot;: true,
      &quot;identity&quot;: {
        &quot;clientId&quot;: &quot;yy&quot;,
        &quot;objectId&quot;: &quot;zz&quot;,
        &quot;resourceId&quot;: &quot;/subscriptions/xx/resourcegroups/MC_xx_SonarQubeCluster_southcentralus/providers/Microsoft.ManagedIdentity/userAssignedIdentities/omsagent-sonarqubecluster&quot;
      }
    }
  },
  &quot;agentPoolProfiles&quot;: [
    {
      &quot;availabilityZones&quot;: [
        &quot;1&quot;
      ],
      &quot;count&quot;: 2,
      &quot;enableAutoScaling&quot;: false,
      &quot;enableEncryptionAtHost&quot;: null,
      &quot;enableFips&quot;: false,
      &quot;enableNodePublicIp&quot;: null,
      &quot;enableUltraSsd&quot;: null,
      &quot;gpuInstanceProfile&quot;: null,
      &quot;kubeletConfig&quot;: null,
      &quot;kubeletDiskType&quot;: &quot;OS&quot;,
      &quot;linuxOsConfig&quot;: null,
      &quot;maxCount&quot;: null,
      &quot;maxPods&quot;: 110,
      &quot;minCount&quot;: null,
      &quot;mode&quot;: &quot;System&quot;,
      &quot;name&quot;: &quot;agentpool&quot;,
      &quot;nodeImageVersion&quot;: &quot;AKSUbuntu-1804gen2containerd-2021.07.25&quot;,
      &quot;nodeLabels&quot;: {},
      &quot;nodePublicIpPrefixId&quot;: null,
      &quot;nodeTaints&quot;: null,
      &quot;orchestratorVersion&quot;: &quot;1.20.7&quot;,
      &quot;osDiskSizeGb&quot;: 128,
      &quot;osDiskType&quot;: &quot;Managed&quot;,
      &quot;osSku&quot;: &quot;Ubuntu&quot;,
      &quot;osType&quot;: &quot;Linux&quot;,
      &quot;podSubnetId&quot;: null,
      &quot;powerState&quot;: {
        &quot;code&quot;: &quot;Running&quot;
      },
      &quot;provisioningState&quot;: &quot;Succeeded&quot;,
      &quot;proximityPlacementGroupId&quot;: null,
      &quot;scaleDownMode&quot;: null,
      &quot;scaleSetEvictionPolicy&quot;: null,
      &quot;scaleSetPriority&quot;: null,
      &quot;spotMaxPrice&quot;: null,
      &quot;tags&quot;: null,
      &quot;type&quot;: &quot;VirtualMachineScaleSets&quot;,
      &quot;upgradeSettings&quot;: null,
      &quot;vmSize&quot;: &quot;Standard_DS2_v2&quot;
    }
  ],
  &quot;apiServerAccessProfile&quot;: {
    &quot;authorizedIpRanges&quot;: null,
    &quot;enablePrivateCluster&quot;: false,
    &quot;enablePrivateClusterPublicFqdn&quot;: null,
    &quot;privateDnsZone&quot;: null
  },
  &quot;autoScalerProfile&quot;: null,
  &quot;autoUpgradeProfile&quot;: null,
  &quot;azurePortalFqdn&quot;: &quot;sonarqubecluster-dns-4b5e95d4.portal.hcp.southcentralus.azmk8s.io&quot;,
  &quot;disableLocalAccounts&quot;: null,
  &quot;diskEncryptionSetId&quot;: null,
  &quot;dnsPrefix&quot;: &quot;SonarQubeCluster-dns&quot;,
  &quot;enablePodSecurityPolicy&quot;: null,
  &quot;enableRbac&quot;: true,
  &quot;extendedLocation&quot;: null,
  &quot;fqdn&quot;: &quot;sonarqubecluster-dns-4b5e95d4.hcp.southcentralus.azmk8s.io&quot;,
  &quot;fqdnSubdomain&quot;: null,
  &quot;httpProxyConfig&quot;: null,
  &quot;id&quot;: &quot;/subscriptions/xx/resourcegroups/RG/providers/Microsoft.ContainerService/managedClusters/SonarQubeCluster&quot;,
  &quot;identity&quot;: {
    &quot;principalId&quot;: &quot;yy&quot;,
    &quot;tenantId&quot;: &quot;rr&quot;,
    &quot;type&quot;: &quot;SystemAssigned&quot;,
    &quot;userAssignedIdentities&quot;: null
  },
  &quot;identityProfile&quot;: {
    &quot;kubeletidentity&quot;: {
      &quot;clientId&quot;: &quot;yy&quot;,
      &quot;objectId&quot;: &quot;zz&quot;,
      &quot;resourceId&quot;: &quot;/subscriptions/xx/resourcegroups/MC_xx_SonarQubeCluster_southcentralus/providers/Microsoft.ManagedIdentity/userAssignedIdentities/SonarQubeCluster-agentpool&quot;
    }
  },
  &quot;kubernetesVersion&quot;: &quot;1.20.7&quot;,
  &quot;linuxProfile&quot;: null,
  &quot;location&quot;: &quot;southcentralus&quot;,
  &quot;maxAgentPools&quot;: 100,
  &quot;name&quot;: &quot;SonarQubeCluster&quot;,
  &quot;networkProfile&quot;: {
    &quot;dnsServiceIp&quot;: &quot;10.0.0.10&quot;,
    &quot;dockerBridgeCidr&quot;: &quot;172.17.0.1/16&quot;,
    &quot;loadBalancerProfile&quot;: {
      &quot;allocatedOutboundPorts&quot;: null,
      &quot;effectiveOutboundIPs&quot;: [
        {
          &quot;id&quot;: &quot;/subscriptions/xx/resourceGroups/MC_xx_SonarQubeCluster_southcentralus/providers/Microsoft.Network/publicIPAddresses/nn&quot;,
          &quot;resourceGroup&quot;: &quot;MC_xx_SonarQubeCluster_southcentralus&quot;
        }
      ],
      &quot;idleTimeoutInMinutes&quot;: null,
      &quot;managedOutboundIPs&quot;: {
        &quot;count&quot;: 1
      },
      &quot;outboundIPs&quot;: null,
      &quot;outboundIpPrefixes&quot;: null
    },
    &quot;loadBalancerSku&quot;: &quot;Standard&quot;,
    &quot;natGatewayProfile&quot;: null,
    &quot;networkMode&quot;: null,
    &quot;networkPlugin&quot;: &quot;kubenet&quot;,
    &quot;networkPolicy&quot;: null,
    &quot;outboundType&quot;: &quot;loadBalancer&quot;,
    &quot;podCidr&quot;: &quot;10.244.0.0/16&quot;,
    &quot;serviceCidr&quot;: &quot;10.0.0.0/16&quot;
  },
  &quot;nodeResourceGroup&quot;: &quot;MC_xx_SonarQubeCluster_southcentralus&quot;,
  &quot;podIdentityProfile&quot;: null,
  &quot;powerState&quot;: {
    &quot;code&quot;: &quot;Running&quot;
  },
  &quot;privateFqdn&quot;: null,
  &quot;privateLinkResources&quot;: null,
  &quot;provisioningState&quot;: &quot;Succeeded&quot;,
  &quot;resourceGroup&quot;: &quot;RG&quot;,
  &quot;securityProfile&quot;: null,
  &quot;servicePrincipalProfile&quot;: {
    &quot;clientId&quot;: &quot;msi&quot;
  },
  &quot;sku&quot;: {
    &quot;name&quot;: &quot;Basic&quot;,
    &quot;tier&quot;: &quot;Free&quot;
  },
  &quot;type&quot;: &quot;Microsoft.ContainerService/ManagedClusters&quot;,
  &quot;windowsProfile&quot;: null
}
</code></pre>
<p>Any idea of what is wrong?</p>
<p>Thank you in advance.</p>

X T
<p>The <code>userAssignedIdentityID</code> in your <code>SecretProviderClass</code> must be the User-assigned Kubelet managed identity ID (Managed Identity for the NodePool) and not the Managed Identity created for your AKS bcs the volumes will be access via kubelet on the nodes.</p>
<pre><code>apiVersion: secrets-store.csi.x-k8s.io/v1alpha1
kind: SecretProviderClass
metadata:
  name: azure-kvname-user-msi
spec:
  provider: azure
  parameters:
    usePodIdentity: &quot;false&quot;
    useVMManagedIdentity: &quot;true&quot;
    userAssignedIdentityID: &quot;&lt;Kubelet identity ID&gt;&quot;
    keyvaultName: &quot;kvname&quot;
</code></pre>
<p>You also need to assign a Role to this Kubelet Identity:</p>
<pre><code>resource &quot;azurerm_role_assignment&quot; &quot;akv_kubelet&quot; {
  scope                = azurerm_key_vault.akv.id
  role_definition_name = &quot;Key Vault Secrets Officer&quot;
  principal_id         = azurerm_kubernetes_cluster.aks.kubelet_identity[0].object_id
}
</code></pre>
<p>or</p>
<pre><code>export KUBE_ID=$(az aks show -g &lt;resource group&gt; -n &lt;aks cluster name&gt; --query identityProfile.kubeletidentity.objectId -o tsv)
export AKV_ID=$(az keyvault show -g &lt;resource group&gt; -n &lt;akv name&gt; --query id -o tsv)
az role assignment create --assignee $KUBE_ID --role &quot;Key Vault Secrets Officer&quot; --scope $AKV_ID
</code></pre>
<p>Documentation can be found <a href="https://azure.github.io/secrets-store-csi-driver-provider-azure/configurations/identity-access-modes/user-assigned-msi-mode/#configure-user-assigned-managed-identity-to-access-keyvault" rel="nofollow noreferrer">here for user-assigned identity</a> and <a href="https://azure.github.io/secrets-store-csi-driver-provider-azure/configurations/identity-access-modes/system-assigned-msi-mode/" rel="nofollow noreferrer">here for system-assigned identity.</a></p>
