<p>I am trying to auto-scale my redis workers based on queue size, I am collecting the metrics using <code>redis_exporter</code> and <code>promethues-to-sd</code> sidecars in my redis deployment as so:</p>

<pre><code>spec:
  containers:
    - name: master
      image: redis
      env:
        - name: MASTER
          value: "true"
      ports:
        - containerPort: 6379
      resources:
        limits:
          cpu: "100m"
        requests:
          cpu: "100m"
    - name: redis-exporter
      image: oliver006/redis_exporter:v0.21.1
      env:
      ports:
        - containerPort: 9121
      args: ["--check-keys=rq*"]
      resources:
        requests:
          cpu: 100m
          memory: 100Mi
    - name: prometheus-to-sd
      image: gcr.io/google-containers/prometheus-to-sd:v0.9.2
      command:
        - /monitor
        - --source=:http://localhost:9121
        - --stackdriver-prefix=custom.googleapis.com
        - --pod-id=$(POD_ID)
        - --namespace-id=$(POD_NAMESPACE)
        - --scrape-interval=15s
        - --export-interval=15s
      env:
        - name: POD_ID
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.uid
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
      resources:
        requests:
          cpu: 100m
          memory: 100Mi
</code></pre>

<p>I can then view the metric (redis_key_size) in Metrics Explorer as:</p>

<pre><code>metric.type="custom.googleapis.com/redis_key_size" 
resource.type="gke_container"
</code></pre>

<p>(I CAN'T view the metric if I change <code>resource.type=k8_pod</code>)</p>

<p>However I can't seem to get the HPA to read in these metrics getting a <code>failed to get metrics error</code>, and can't seem to figure out the correct <code>Object</code> definition. </p>

<p>I've tried both <code>.object.target.kind=Pod</code> and <code>Deployment</code>, with deployment I get the additional error <code>"Get namespaced metric by name for resource \"deployments\"" is not implemented</code>.</p>

<p>I don't know if this issue is related to the <code>resource.type="gke_container"</code> and how to change that?</p>

<pre><code>apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: {{ template "webapp.backend.fullname" . }}-workers
  namespace: default
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: {{ template "webapp.backend.fullname" . }}-workers
  minReplicas: 1
  maxReplicas: 4
  metrics:
    - type: Object
      object:
        target:
          kind: &lt;not sure&gt;
          name: &lt;not sure&gt;
        metricName: redis_key_size
        targetValue: 4
</code></pre>

<p>--- Update ---</p>

<p>This works if I use <code>kind: Pod</code> and manually set <code>name</code> to the pod name created by the deployment, however this is far from perfect.</p>

<p>I also tried this setup using type <code>Pods</code>, however the HPA says it can't read the metrics <code>horizontal-pod-autoscaler  failed to get object metric value: unable to get metric redis_key_size: no metrics returned from custom metrics API</code></p>

<pre><code>apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: {{ template "webapp.backend.fullname" . }}-workers
  namespace: default
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: {{ template "webapp.backend.fullname" . }}-workers
  minReplicas: 1
  maxReplicas: 4
  metrics:
  - type: Pods
    pods:
      metricName: redis_key_size
      targetAverageValue: 4
</code></pre>

<p>As a workaround for deployments it appears that the metrics have to be exported from pods IN the target deployment. </p>

<p>To get this working I had to move the <code>prometheus-to-sd</code> container to the deployment I wanted to scale and then scrape the exposed metrics from Redis-Exporter in the Redis deployment via the Redis service, exposing 9121 on the Redis service, and changing the CLA for the the <code>prometheus-to-sd</code> container such that:</p>

<p><code>- --source=:http://localhost:9121</code> -> - <code>--source=:http://my-redis-service:9121</code></p>

<p>and then using the HPA</p>

<pre><code>apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: {{ template "webapp.backend.fullname" . }}-workers
  namespace: default
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: {{ template "webapp.backend.fullname" . }}-workers
  minReplicas: 1
  maxReplicas: 4
  metrics:
  - type: Pods
    pods:
      metricName: redis_key_size
      targetAverageValue: 4
</code></pre>
