<p>I used the following metrics inside hpa</p>

<pre><code>apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
  name: app-svc-hpa
  namespace: default
spec:
  scaleTargetRef:
    apiVersion: extensions/v1beta1
    kind: Deployment
    name: app-svc
  minReplicas: 1
  maxReplicas: 1000
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 50
  - type: Pods
    pods:
      metric:
        name: packets-per-second
      target:
        type: AverageValue
        averageValue: 1k
</code></pre>

<p>But the hpa is unable to get the metrics</p>

<p><code>Warning  FailedGetPodsMetric           14s (x6 over 1m)  horizontal-pod-autoscaler  unable to get metric packets-per-second: unable to fetch metrics from custom metrics API: the server could not find the descriptor for metric custom.googleapis.com/packets-per-second: googleapi: Error 403: Permission monitoring.metricDescriptors.get denied (or the resource may not exist)., forbidden</code></p>

<p>I am running the pods on a dedicated nodepool and each nodes is running under a service account.</p>

<p>The service account does have these iam roles </p>

<p>Monitoring Viewere,
Monitoring Metrics Writer</p>

<p>Unsure how to fix this error. Any pointers are greatly appreciated. Thanks.</p>

<p>I had a cluster with <code>workload identity</code> enabled. Apparently when a cluster has workload identity enabled, the metrics fetch was failing. </p>

<p>1) I had to install the custom stack driver adapter and create the custom metric as pointed by David Kruk in his comments</p>

<p>2) I had to add the <code>hostNetwork:true</code> in the custom stackdriver adapter deployment pod spec. The issue is mentioned <a href="https://github.com/GoogleCloudPlatform/k8s-stackdriver/issues/315" rel="nofollow noreferrer">here</a> in github repository for csa</p>

<p>With these two updates, the autoscaler works as expected.</p>
