<p>I wrote a python script to dump my database, gzip it and move it to a cloud storage.</p>
<p>Locally everything works smooth and consumses basically no memory (max 20MB RAM, CPU is quite high, I/O is at max).</p>
<p><strong>When I run it as a job in my kubernetes cluster, the memory usage piles up to about 1.6 GB.</strong></p>
<p>That is more or less the size of my gzipped dumpfile.</p>
<p>Here's my dumping logic:</p>
<pre><code>mysqldump_command = ['mysqldump', f'--host={host}', f'--port={port}', f'--user={username}',
                         f'--password={password}', '--databases', '--compact',
                         '--routines', db_name, f'--log-error={self.errorfile}']

print(f'## Creating mysql dump')        
with open(self.filename_gzip, 'wb', 0) as f:
    p1 = subprocess.Popen(mysqldump_command, stdout=subprocess.PIPE)
    p2 = subprocess.Popen('gzip', stdin=p1.stdout, stdout=f)
p1.stdout.close()  # force write error (/SIGPIPE) if p2 dies
p2.wait()
p1.wait()
</code></pre>
<p>I tried:</p>
<ul>
<li>setting <code>PYTHONUNBUFFERED=1</code>, no effect.</li>
<li>I tried <a href="https://stackoverflow.com/a/24103662/1331671">this logic</a> but it was even worse</li>
<li>I tried creating the dump first as a file and gzip it afterwards, was the worst of all my experiments</li>
</ul>
<p>Any further ideas?</p>

<p>I reproduced your setup and the dump still works with more restrictive limits.</p>

<p>I just set the ressource caps in the my-job.yml and it does not consume much memory any more:</p>

<pre><code>resources:
  limits:
    memory: "100Mi"
  requests:
    memory: "50Mi"
</code></pre>
