<p>I am having an issue when trying to setup <strong>Virtual Nodes</strong> for <strong>Azure Kubernetes cluster</strong> using <strong>Terraform</strong>.</p>
<p>When I check the pod for the aci-connector-linux, I get the below error:</p>
<pre><code>Events:
  Type     Reason   Age                     From     Message
  ----     ------   ----                    ----     -------
  Normal   Pulled   41m (x50 over 4h26m)    kubelet  Container image &quot;mcr.microsoft.com/oss/virtual-kubelet/virtual-kubelet:1.4.1&quot; already present on machine
  Warning  BackOff  68s (x1222 over 4h26m)  kubelet  Back-off restarting failed container
</code></pre>
<p>I've also granted the System Assigned identity of the Azure Kubernetes Cluster the required contributor role using the documentation here - <a href="https://github.com/terraform-providers/terraform-provider-azurerm/blob/master/examples/kubernetes/aci_connector_linux/main.tf" rel="nofollow noreferrer">https://github.com/terraform-providers/terraform-provider-azurerm/blob/master/examples/kubernetes/aci_connector_linux/main.tf</a> but I'm still getting CrashLoopBackOff status error.</p>

Promise Preston
<p>I finally fixed it.</p>
<p>The issue was caused by the Outdated documentation for <code>aci-connector-linux</code> here - <a href="https://github.com/terraform-providers/terraform-provider-azurerm/blob/master/examples/kubernetes/aci_connector_linux/main.tf" rel="nofollow noreferrer">https://github.com/terraform-providers/terraform-provider-azurerm/blob/master/examples/kubernetes/aci_connector_linux/main.tf</a> which assigns the role to the Managed identity of the Azure Kubernetes cluster</p>
<p><strong>Here's how I fixed it</strong>:</p>
<p>Azure Kubernetes Service creates Node resource group which is separate from the resource group for the Kubernetes Cluster. Within the Node resource group, AKS creates a Managed Identity for the <code>aci-connector-linux</code>. The name of the Node resource group is usually <code>MC_&lt;KubernetesResourceGroupName_KubernetesServiceName-KubernetesResourceGroupLocation&gt;</code>, so if your <strong>KubernetesResourceGroupName</strong> is <code>MyResourceGroup</code> and if the <strong>KubernetesServiceName</strong> is <code>my-test-cluster</code> and if the <strong>KubernetesResourceGroupLocation</strong> <code>westeurope</code>, then the Node resource group will be <code>MC_MyResourceGroup_my-test-cluster_westeurope</code>. You can view the resources in the Azure Portal under Resource Groups.</p>
<p>Next, you can view the root cause of the issue by viewing the logs of the <code>aci-connector-linux</code> pod using the command:</p>
<pre><code>kubectl logs aci-connector-linux-577bf54d75-qm9kl -n kube-system
</code></pre>
<p>And you will an output like this:</p>
<blockquote>
<p>time=&quot;2022-06-29T15:23:38Z&quot; level=fatal msg=&quot;error initializing provider azure: error setting up network profile: error while looking up subnet: api call to <a href="https://management.azure.com/subscriptions/0237fb7-7530-43ba-96ae-927yhfad80d1/resourcegroups/MyResourceGroup/providers/Microsoft.Network/virtualNetworks/my-vnet/subnets/k8s-aci-node-pool-subnet?api-version=2018-08-01" rel="nofollow noreferrer">https://management.azure.com/subscriptions/0237fb7-7530-43ba-96ae-927yhfad80d1/resourcegroups/MyResourceGroup/providers/Microsoft.Network/virtualNetworks/my-vnet/subnets/k8s-aci-node-pool-subnet?api-version=2018-08-01</a>: got HTTP response status code 403 error code &quot;AuthorizationFailed&quot;: The client '560df3e9b-9f64-4faf-aa7c-6tdg779f81c7' with object id '560df3e9b-9f64-4faf-aa7c-6tdg779f81c7' does not have authorization to perform action 'Microsoft.Network/virtualNetworks/subnets/read' over scope '/subscriptions/0237fb7-7530-43ba-96ae-927yhfad80d1/resourcegroups/MyResourceGroup/providers/Microsoft.Network/virtualNetworks/my-vnet/subnets/k8s-aci-node-pool-subnet' or the scope is invalid. If access was recently granted, please refresh your credentials.&quot;</p>
</blockquote>
<p>You can fix this in <strong>Terraform</strong> using the code below:</p>
<pre><code># Get subnet ID
data &quot;azurerm_subnet&quot; &quot;k8s_aci&quot; {
  name                 = &quot;k8s-aci-node-pool-uat-subnet&quot;
  virtual_network_name = &quot;sparkle-uat-vnet&quot;
  resource_group_name  = data.azurerm_resource_group.main.name
}

# Get the Identity of a service principal
data &quot;azuread_service_principal&quot; &quot;aks_aci_identity&quot; {
  display_name = &quot;aciconnectorlinux-${var.kubernetes_cluster_name}&quot;
  depends_on = [module.kubernetes_service_uat]
}

# Assign role to aci identity
module &quot;role_assignment_aci_nodepool_subnet&quot; {
  source = &quot;../../../modules/azure/role-assignment&quot;

  role_assignment_scope        = data.azurerm_subnet.k8s_aci.id
  role_definition_name         = var.role_definition_name.net-contrib
  role_assignment_principal_id = data.azuread_service_principal.aks_aci_identity.id
}
</code></pre>
<p>You can also achieve this using the Azure CLI command below:</p>
<pre><code>az role assignment create --assignee &lt;Object (principal) ID&gt; --role &quot;Network Contributor&quot; --scope &lt;subnet-id&gt;
</code></pre>
<p><strong>Note</strong>: The <strong>Object (principal) ID</strong> is the ID that you obtained in the error message.</p>
<p>An example is this:</p>
<pre><code>az role assignment create --assignee 560df3e9b-9f64-4faf-aa7c-6tdg779f81c7 --role &quot;Network Contributor&quot; --scope /subscriptions/0237fb7-7530-43ba-96ae-927yhfad80d1/resourcegroups/MyResourceGroup/providers/Microsoft.Network/virtualNetworks/my-vnet/subnets/k8s-aci-node-pool-subnet
</code></pre>
<p><strong>Resources</strong>:</p>
<p><a href="https://github.com/hashicorp/terraform-provider-azurerm/issues/9733" rel="nofollow noreferrer">Aci connector linux should export the identity associated to its addon</a></p>
<p><a href="https://github.com/Azure/AKS/issues/1894" rel="nofollow noreferrer">Using Terraform to create an AKS cluster with &quot;SystemAssigned&quot; identity and aci_connector_linux profile enabled does not result in a creation of a virtual node</a></p>
<p><a href="https://cloud.netapp.com/blog/azure-cvo-blg-azure-kubernetes-service-tutorial-integrate-aks-with-aci" rel="nofollow noreferrer">Azure Kubernetes Service Tutorial: How to Integrate AKS with Azure Container Instances</a></p>
<p><a href="https://docs.microfocus.com/doc/SMAX/2021.05/AKSFailToPatchLB" rel="nofollow noreferrer">Fail to configure a load balancer (AKS)</a></p>
