<p>Problem: the code we're running on Kubernetes Pods have a very high variance across it's runtime; specifically, it has occasional CPU &amp; Memory <em>spikes</em> when certain conditions are triggered. These triggers involve user queries with hard realtime requirements (system has to respond within &lt;5 seconds). </p>

<p>Under conditions where the node serving the spiking pod doesn't have enough CPU/RAM, Kubernetes responds to these excessive requests by killing the pod altogether; which results in <em>no output</em> across any time whatsoever.</p>

<p>In what way can we ensure, that these spikes are being taken into account when pods are allocated; and more critically, that no pod shutdown happens for these reasons?</p>

<p>Thanks!</p>

<p>High availability of pods with load can be achieved in two ways:</p>

<blockquote>
  <p><strong>Configuring More CPU/Memory</strong></p>
</blockquote>

<p>As the applications requires more CPU/memory during the peak times configure in such a way that allocated resources for the POD will take care of extra load. Configure the POD something like this:</p>

<pre><code>resources:
      requests:
        memory: "64Mi"
        cpu: "250m"
      limits:
        memory: "128Mi"
        cpu: "500m"
</code></pre>

<p>You can increase the limits based on the usage. But this way of doing can cause two issues</p>

<p>1) <strong>Underutilized resources</strong></p>

<p>As the resources are allocated in large number, these may go wasted unless there is a spike in the traffic.</p>

<p>2) <strong>Deployment failure</strong></p>

<p>POD deployment may fail because of not having enough resources in the kubernetes node to cater the request.</p>

<p>For more info : <a href="https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/" rel="nofollow noreferrer">https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/</a></p>

<p><strong>> Autoscaling</strong></p>

<p>Ideal way of doing it is to autoscale the POD based on the traffic.</p>

<pre><code>kubectl autoscale deployment &lt;DEPLOY-APP-NAME&gt; --cpu-percent=50 --min=1 --max=10 
</code></pre>

<p>Configure the cpu-percent based on the requirement, else 80% by default. Min and max are the number of PODS which can be configured accordingly. </p>

<p>So each time a POD hits the CPU percent with 50% a new pod will be launched and continues till it launches a max of 10 PODS and same applicable for vice-versa scenario.</p>

<p>For more info: <a href="https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/" rel="nofollow noreferrer">https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/</a></p>
