<p>I have a use case where in my jax-rs rest api exposed will do a memory intensive operation like generating a pdf and sending it as response. I want to check pod memory usage before start doing the business logic  and if the  current memory usage is &gt; 50% , i would like to send a error response to user asking him to try again later. how do i check kubernetes pod memory usage inside my rest api.  Is it even possible?</p>
<p>My current code somewhat looks like below.</p>
<pre><code>@Path(&quot;/doSomeMemoryStuff&quot;)
@Produces(&quot;text/plain&quot;)
public Response doStuff(){

     int memoryUsage = getPodCurrentMemoryUsage();  //Get pod current memory usage
     if( memoryUsage &lt;= 50 ) {

          //do Some memory intensive operation
      }
      else{
           //memory usage is more than 50 % , return error
           return Response.ok(&quot;Try again later.&quot;).build();
          }
  }         
</code></pre>
<p>How do I find out the current memory usage of pod from my rest api.  Thanks in advance.</p>

Stunner
<p>Strictly, we could paraphrase your statement as <em>&quot;Kubernetes pod memory usage automatically&quot;</em> in the sense that if you have processes that, from time to time, consume a lot of resources that need to be freed, you can use <strong>Vertical Pod Autoscaler</strong>.</p>
<p>Some interesting references:</p>
<ol>
<li><a href="https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler" rel="nofollow noreferrer">https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler</a></li>
<li><a href="https://docs.aws.amazon.com/eks/latest/userguide/vertical-pod-autoscaler.html" rel="nofollow noreferrer">https://docs.aws.amazon.com/eks/latest/userguide/vertical-pod-autoscaler.html</a></li>
<li><a href="https://cloud.google.com/kubernetes-engine/docs/concepts/verticalpodautoscaler" rel="nofollow noreferrer">https://cloud.google.com/kubernetes-engine/docs/concepts/verticalpodautoscaler</a></li>
</ol>
<p>The use of VPA is appropriate when resource requirements are very different for one process or another.</p>
<p>If this is not the case (more or less all requests will use a known amount of resources) or even if you use VPA, it is recommended that you limit your services to not accept requests if they are working on an expensive operation, Kubernetes will automatically increase or decrease the number of Pods depending on the load and your users will receive a 503 error which is precisely to indicate that they cannot be served now and should try again later.</p>
<p>That is to say:</p>
<ol>
<li>do not use VPA if not strictly necessary.</li>
<li>configure your deployment with an adequate number of pods.</li>
<li>restrict your services within the pods to a single concurrent request (or as many as fit in your resource configuration).</li>
<li>don't do anything special, if your system has reached the limit you have set, just let the users receive a 503 (your user interface will translate the error as <em>&quot;Try again later&quot;</em>).</li>
</ol>
<p>The details of a deployment may vary, but basically by acting at three levels, you can give your infrastructure some adaptability to the type of load:</p>
<ol>
<li><strong>Application Level:</strong> for every application instance, you can define the (http requests) rate limit. It must be aligned with the Requests/Limits of your Pods. If you cannot modify your applications (i.e. using <a href="https://github.com/vladimir-bukhtoyarov/bucket4j" rel="nofollow noreferrer">bucket4j</a>) you can add an <a href="https://www.magalix.com/blog/the-adapter-pattern" rel="nofollow noreferrer">Adapter</a> to your Pod using (e.g.) Nginx (see point 3 for specific configuration).</li>
<li><strong>Deployment Level:</strong> once your application will not break due to request overload, you should be able to scale your infrastructure, horizontally with Requests/Limits for each Pod or from replicas in your Deployment using <a href="https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/" rel="nofollow noreferrer">https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/</a> (there are a lot of metrics that can be used for auto-scaling).</li>
<li><strong>Load Balancer Level:</strong> for simple scenarios you can simply configure an ingress controller for rate limit (i.e. using <a href="https://medium.com/titansoft-engineering/rate-limiting-for-your-kubernetes-applications-with-nginx-ingress-2e32721f7f57" rel="nofollow noreferrer">Nginx controller</a>) but, if you are able to segment your requests (e.g. <code>...?queryType=hard&amp;...</code>), you can segregate your configuration (points 1 and 2), to maintain multiple infrastructures (with several vertical scalings), each one already prepared to handle specific types of requests, this can be easily done with Nginx (<a href="https://istio.io/latest/docs/tasks/traffic-management/locality-load-balancing/failover/" rel="nofollow noreferrer">Istio</a> might be overkill).</li>
</ol>
<p>With this strategy, suppose you have two zones: <em>&quot;LR: Low Resources&quot;</em> and <em>&quot;HR: High Resources&quot;</em>. If there is no load on your system, neither of the two zones consume resources (i.e. <code>minReplies: 1</code>), if there are many <em>&quot;LR&quot;</em> requests the resources are used in this zone, if they are in <em>&quot;HR&quot;</em> they are used in this other one, if they are in both, they are distributed between both. Logically the maximum load will be <code>LR.maxReplies + HR.maxReplies</code> (you can make more complex rules e.g. using Istio but always use the simplest scheme that you think will work for you).</p>
