<p>Getting error while configuring prometheus in azure kubernetes</p>
<p><a href="https://i.stack.imgur.com/8hbaD.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/8hbaD.jpg" alt="enter image description here" /></a></p>

Punit kumar
<p><em>I tried to reproduce the same issue in my environment and got the below results</em></p>
<p><em>I have the cluster and I am trying to configure the Prometheus in azure Kubernetes and I got the successful deployment</em></p>
<p><img src="https://i.stack.imgur.com/U7EOF.png" alt="enter image description here" /></p>
<p><em>To verify the agent is deployed or not use the below commands</em></p>
<pre><code> kubectl get ds &lt;dep_name&gt; --namespace=kube-system
 kubectl get rrs --namespace=kube-system
</code></pre>
<p><img src="https://i.stack.imgur.com/PUfvM.png" alt="enter image description here" /></p>
<p><em>This error getting because of you are using the service principal instead of managed identity</em></p>
<p><em>For enabling the managed identity please follow the below commands</em></p>
<p><em>AKS cluster with service principal first disable monitoring and then upgrade to managed identity, the azure public cloud is supporting  for this migration</em></p>
<p><em>To get the log analytics workspace id</em></p>
<pre><code>az aks show -g &lt;rg_name&gt; -n &lt;cluster_name&gt; | grep -i &quot;logAnalyticsWorkspaceResourceID&quot;
</code></pre>
<p>For disable the monitoring use the below command</p>
<pre><code>az aks disable-addons -a monitoring -g &lt;rg_name&gt; -n &lt;cluster_name&gt;
</code></pre>
<p><em>Or I can get it on portal in the azure monitor logs</em></p>
<p><img src="https://i.stack.imgur.com/wmByq.png" alt="enter image description here" /></p>
<p><em>I have upgrade the cluster to system managed identity, use the below command to upgrade</em></p>
<pre><code>az aks update -g &lt;rg_name&gt; -n &lt;cluster_name&gt; --enable-managed-identity
</code></pre>
<p><img src="https://i.stack.imgur.com/96Zo8.png" alt="enter image description here" /></p>
<p><em>I have enable the monitoring addon with the managed identity authentication</em></p>
<pre><code>az aks enable-addons -a monitoring --enable-msi-auth-for-monitoring -g &lt;rg_name&gt; -n &lt;cluster_name&gt; --workspace-resource-id &lt;workspace_resource_id&gt;
</code></pre>
<p><em>For more information use this document for <a href="https://learn.microsoft.com/en-us/azure/azure-monitor/containers/container-insights-enable-aks?tabs=azure-cli#migrate-to-managed-identity-authentication" rel="nofollow noreferrer">Reference</a></em></p>
