<p>Kubernetes v1.19 in AWS EKS</p>
<p>I'm trying to implement horizontal pod autoscaling in my EKS cluster, and am trying to mimic what we do now with ECS. With ECS, we do something similar to the following</p>
<ul>
<li>scale up when CPU &gt;= 90% after 3 consecutive 1-min periods of sampling</li>
<li>scale down when CPU &lt;= 60% after 5 consecutive 1-min periods of sampling</li>
<li>scale up when memory &gt;= 85% after 3 consecutive 1-min periods of sampling</li>
<li>scale down when memory &lt;= 70% after 5 consecutive 1-min periods of sampling</li>
</ul>
<p>I'm trying to use the <code>HorizontalPodAutoscaler</code> kind, and <code>helm create</code> gives me this template. (Note I modified it to suit my needs, but the <code>metrics</code> stanza remains.)</p>
<pre><code>{- if .Values.autoscaling.enabled }}
apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: {{ include &quot;microserviceChart.Name&quot; . }}
  labels:
    {{- include &quot;microserviceChart.Name&quot; . | nindent 4 }}
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: {{ include &quot;microserviceChart.Name&quot; . }}
  minReplicas: {{ include &quot;microserviceChart.minReplicas&quot; . }}
  maxReplicas: {{ include &quot;microserviceChart.maxReplicas&quot; . }}
  metrics:
    {{- if .Values.autoscaling.targetCPUUtilizationPercentage }}
    - type: Resource
      resource:
        name: cpu
        targetAverageUtilization: {{ .Values.autoscaling.targetCPUUtilizationPercentage }}
    {{- end }}
    {{- if .Values.autoscaling.targetMemoryUtilizationPercentage }}
    - type: Resource
      resource:
        name: memory
        targetAverageUtilization: {{ .Values.autoscaling.targetMemoryUtilizationPercentage }}
    {{- end }}
{{- end }}
</code></pre>
<p>However, how do I fit the scale up/down information shown in <a href="https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/" rel="noreferrer">Horizontal Pod Autoscaling</a> in the above template, to match the behavior that I want?</p>

<p>The Horizontal Pod Autoscaler automatically scales the number of Pods in a replication controller, deployment, replica set or stateful set based on observed metrics (like <code>CPU</code> or <code>Memory</code>).</p>
<p>There is an official walkthrough focusing on <code>HPA</code> and it's scaling:</p>
<ul>
<li><em><a href="https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/" rel="noreferrer">Kubernetes.io: Docs: Tasks: Run application: Horizontal pod autoscale: Walkthrough</a></em></li>
</ul>
<hr />
<p>The algorithm that scales the amount of replicas is the following:</p>
<ul>
<li><code>desiredReplicas = ceil[currentReplicas * ( currentMetricValue / desiredMetricValue )]</code></li>
</ul>
<p>An example (of already rendered) autoscaling can be implemented with a <code>YAML</code> manifest like below:</p>
<pre><code>apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
  name: HPA-NAME
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: DEPLOYMENT-NAME
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 75
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 75
</code></pre>
<blockquote>
<p><strong>A side note!</strong></p>
<p><code>HPA</code> will use calculate both metrics and chose the one with bigger <code>desiredReplicas</code>!</p>
</blockquote>
<p>Addressing a comment I wrote under the question:</p>
<blockquote>
<p>I think we misunderstood each other. It's perfectly okay to &quot;scale up when CPU &gt;= 90&quot; but due to logic behind the formula I don't think it will be possible to say &quot;scale down when CPU &lt;=70&quot;. According to the formula it would be something in the midst of: scale up when CPU &gt;= 90 and scale down when CPU =&lt; 45.</p>
</blockquote>
<p><strong>This example could be misleading and not 100% true in all scenarios.</strong> Taking a look on following example:</p>
<ul>
<li><code>HPA</code> set to <code>averageUtilization</code> of <code>75%</code>.</li>
</ul>
<p>Quick calculations with some degree of approximation (default tolerance for <code>HPA</code> is <code>0.1</code>):</p>
<ul>
<li><code>2</code> replicas:
<ul>
<li><code>scale-up</code> (by <code>1</code>) should happen when: <code>currentMetricValue</code> is &gt;=<code>80%</code>:
<ul>
<li><code>x = ceil[2 * (80/75)]</code>, <code>x = ceil[2,1(3)]</code>, <code>x = 3</code></li>
</ul>
</li>
<li><code>scale-down</code> (by <code>1</code>) should happen when <code>currentMetricValue</code> is &lt;=<code>33%</code>:
<ul>
<li><code>x = ceil[2 * (33/75)]</code>, <code>x = ceil[0,88]</code>, <code>x = 1</code></li>
</ul>
</li>
</ul>
</li>
<li><code>8</code> replicas:
<ul>
<li><code>scale-up</code> (by <code>1</code>) should happen when <code>currentMetricValue</code> is &gt;=<code>76%</code>:
<ul>
<li><code>x = ceil[8 * (76/75)]</code>, <code>x = ceil[8,10(6)]</code>, <code>x = 9</code></li>
</ul>
</li>
<li><code>scale-down</code> (by <code>1</code>) should happen when <code>currentMetricValue</code> is &lt;=<code>64%</code>:
<ul>
<li><code>x = ceil[8 * (64/75)]</code>, <code>x = ceil[6,82(6)]</code>, <code>x = 7</code></li>
</ul>
</li>
</ul>
</li>
</ul>
<p>Following this example, having <code>8</code> replicas with their <code>currentMetricValue</code> at <code>55</code> (<code>desiredMetricValue</code> set to <code>75</code>) should <code>scale-down</code> to <code>6</code> replicas.</p>
<p>More information that describes the decision making of <code>HPA</code> (<strong>for example why it's doesn't scale</strong>) can be found by running:</p>
<ul>
<li><code>$ kubectl describe hpa HPA-NAME</code></li>
</ul>
<pre><code>Name:                                                     nginx-scaler
Namespace:                                                default
Labels:                                                   &lt;none&gt;
Annotations:                                              &lt;none&gt;
CreationTimestamp:                                        Sun, 07 Mar 2021 22:48:58 +0100
Reference:                                                Deployment/nginx-scaling
Metrics:                                                  ( current / target )
  resource memory on pods  (as a percentage of request):  5% (61903667200m) / 75%
  resource cpu on pods  (as a percentage of request):     79% (199m) / 75%
Min replicas:                                             1
Max replicas:                                             10
Deployment pods:                                          5 current / 5 desired
Conditions:
  Type            Status  Reason              Message
  ----            ------  ------              -------
  AbleToScale     True    ReadyForNewScale    recommended size matches current size
  ScalingActive   True    ValidMetricFound    the HPA was able to successfully calculate a replica count from cpu resource utilization (percentage of request)
  ScalingLimited  False   DesiredWithinRange  the desired count is within the acceptable range
Events:
  Type     Reason                   Age                   From                       Message
  ----     ------                   ----                  ----                       -------
  Warning  FailedGetResourceMetric  4m48s (x4 over 5m3s)  horizontal-pod-autoscaler  did not receive metrics for any ready pods
  Normal   SuccessfulRescale        103s                  horizontal-pod-autoscaler  New size: 2; reason: cpu resource utilization (percentage of request) above target
  Normal   SuccessfulRescale        71s                   horizontal-pod-autoscaler  New size: 4; reason: cpu resource utilization (percentage of request) above target
  Normal   SuccessfulRescale        71s                   horizontal-pod-autoscaler  New size: 5; reason: cpu resource utilization (percentage of request) above target
</code></pre>
<hr />
<p><code>HPA</code> scaling procedures can be modified by the changes introduced in Kubernetes version <code>1.18</code> and newer where the:</p>
<blockquote>
<h3>Support for configurable scaling behavior</h3>
<p>Starting from <a href="https://github.com/kubernetes/enhancements/blob/master/keps/sig-autoscaling/20190307-configurable-scale-velocity-for-hpa.md" rel="noreferrer">v1.18</a> the <code>v2beta2</code> API allows scaling behavior to be configured through the HPA <code>behavior</code> field. Behaviors are specified separately for scaling up and down in <code>scaleUp</code> or <code>scaleDown</code> section under the <code>behavior</code> field. A stabilization window can be specified for both directions which prevents the flapping of the number of the replicas in the scaling target. Similarly specifying scaling policies controls the rate of change of replicas while scaling.</p>
<p><em><a href="https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#support-for-configurable-scaling-behavior" rel="noreferrer">Kubernetes.io: Docs: Tasks: Run application: Horizontal pod autoscale: Support for configurable scaling behavior</a></em></p>
</blockquote>
<p>I'd reckon you could used newly introduced field like <code>behavior</code> and <code>stabilizationWindowSeconds</code> to tune your workload to your specific needs.</p>
<p>I also do recommend reaching out to <code>EKS</code> documentation for more reference, support for metrics and examples.</p>
