<p>I have a simple Spring Boot application, deployed on k8s (2 pods).
Briefly described this application takes messages from producer and process them to the consumer. Nothing complicated. </p>

<p>UPD: </p>

<ul>
<li>java version: 1.8.172</li>
<li>javaMemoryOpts: -Xmx2048m -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap</li>
</ul>

<p><a href="https://i.stack.imgur.com/tgbcA.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/tgbcA.png" alt="memory monitoring"></a>
This's a memory consumption for one of 2 podes.</p>

<ul>
<li>blue line - requested memory by k8s</li>
<li>orange line - working set</li>
<li>green line - service usage </li>
<li>yellow line - memory limit by k8s</li>
</ul>

<p>The issue is the high memory usage despite of simplicity of service.
I'd profiled the app, but it seems ok with service: just about 60 threads, no memory leaks and so on. </p>

<p>The used memory never exceed k8s limit even if it's very close to it (no OOM).
Of course, I can add more pods and consumption will become even but I think it's not the right way.</p>

<p>One thing confusing me why used memory always above required even at the start.</p>

<p>Actually, I don't know what's going wrong with it. Does anybody have ideas or maybe know how to reduce app's memory usage?</p>

yevtsy
<p>The answer from mk_sta is helpful and all the information you need is probably in those documents, but I feel it is worth summarising the main points in a full answer.</p>

<p>Your -Xmx2048m (2Gb) option is setting the maximum heap size, but the application will use more memory than this - the Metaspace, the Garbage Collector and lots of other overhead (this is "off heap" memory).</p>

<p>No matter how simple your App, Java will use the available heap size. A 3 line app outputting random Strings, if given a 2Gb heap, will eventually use it all. So it doesn't matter if your Spring Boot app is "simple" - the heap will grow until it reaches the maximum and then you'll see a garbage collection - these are the jagged teeth in your green line.</p>

<p>So these two things together probably explains why you're seeing an upper bound of about 3.8Gb on memory usage.</p>

<p>The graphs you're plotting are probably showing a well behaved application, so stop worrying about memory leaks. I can't tell from the picture if these are minor or major collections. Ie I can't infer from the picture how small you can risk shrinking the Xmx.</p>

<p>Although you say your Spring Boot app is "simple" there's no way of knowing  without seeing its pom how complex it really is. But the document linked from mk_ska here...</p>

<p><a href="https://github.com/dsyer/spring-boot-memory-blog/blob/master/cf.md" rel="nofollow noreferrer">https://github.com/dsyer/spring-boot-memory-blog/blob/master/cf.md</a></p>

<p>...is a very useful one because it shows some good defaults for "small" Spring Boot apps - eg an app using Freemarker to generate non-static content can run happily with a 32Mb heap.</p>

<p>So the easy answer is to try shrinking your -Xmx down to as small as you can before you see garbage collection thrashing. You too might be able to get it down to as little as 32Mb. </p>

<p>Then, you can use these finding to set some sensible values for the resource limit and resource request in your K8S manifests. You'll need a lot more than the eg 32Mb of the heap size - as stated in that doc, Spring boot apps are much happier with container memory of 512Mb or 1Gb. You might get away with 256Mb, but that is tight.</p>
