<p>I have a Python process that does some heavy computations with Pandas and such - not my code so I basically don't have much knowledge on this.</p>
<p>The situation is this Python code used to run perfectly fine on a server with 8GB of RAM maxing all the resources available.</p>
<p>We moved this code to Kubernetes and we can't make it run: even increasing the allocated resources up to 40GB, this process is greedy and will inevitably try to get as much as it can until it gets over the container limit and gets killed by Kubernetes.</p>
<p>I know this code is probably suboptimal and needs rework on its own.</p>
<p>However my question is how to get Docker on Kubernetes mimic what Linux did on the server: give as much as resources as needed by the process without killing it?</p>

<p>I found out that running something like this seems to work:</p>
<pre class="lang-py prettyprint-override"><code>import resource
import os

if os.path.isfile('/sys/fs/cgroup/memory/memory.limit_in_bytes'):
    with open('/sys/fs/cgroup/memory/memory.limit_in_bytes') as limit:
        mem = int(limit.read())
        resource.setrlimit(resource.RLIMIT_AS, (mem, mem))
</code></pre>
<p>This reads the memory limit file from cgroups and set it as both hard and soft limit for the process' max area address space.</p>
<p>You can test by runnning something like:</p>
<pre><code>docker run --it --rm -m 1G --cpus 1 python:rc-alpine
</code></pre>
<p>And then trying to allocate 1G of ram before and after running the script above.</p>
<p>With the script, you'll get a <code>MemoryError</code>, without it the container will be killed.</p>
