<p>I have a very simple query, in a very simple table:</p>
<pre class="lang-py prettyprint-override"><code>def get_defaults(domain):
    defaults = BaseDefaults.query.filter_by(domain=domain).first()
    return defaults
</code></pre>
<p>The table has (no kidding) 3 rows, with 34 columns, 1 PK, 1 Unique, 2 FK. of the following types:</p>
<pre><code>Timestamp: 2 Cols
Integer: 5 Cols
Booleans: 8 Cols
VarChar(100) &amp; Varchar(250): 19 Cols
</code></pre>
<p>Usually, around 5-8 of these columns have null values.
This function runs in the context of a Celery task, which was exploding in memory (1.2Gb), while another task that's running uses around 110Mb.</p>
<p>So I profiled the code with <code>memory_profiler</code> and ends up that this specific function <code>defaults = get_defaults(domain)</code> is eating around 800-900Mb of RAM, which makes absolutely no sense.</p>
<p>I don't see the same behavior locally - it only happens within a Kubernetes Cluster (hosted on DigitalOcean), so it's been hard to understand what might be going on.</p>
<p>The function queries a Postgres DB hosted on RDS, which seems to be working fine (queries from local pc, or from SQL Client work just fine less than 200ms).</p>
<p>I have also found that the bulk of the high memory consumption happens during the first time the query is run, on the second time the query runs (another task run), the same line increases usage by about 120Mb, which seems more plausible. So might be the case that the session management in Flask-SQLAlchemy within Celery is getting in the way also.</p>
<p>It's consuming a great deal of RAM within the cluster and threatening a node's health. If I limit the pod consumption to 512Mb, the pods get created but dies right afterward.</p>
<p>Any ideas on how to troubleshoot, optimize, fix this issue?</p>

<p>For anyone that's going through something similar, I fixed my problem with <code>lazy=True</code> in the <code>backref</code> model declaration.</p>
<p>This wasn't a problem until a completely different table in the database started to grow fast - we were using <code>lazy='joined'</code> which would automatically join every table that had relationships declared with <code>BsaeDefaults</code>.</p>
<p>By using <code>lazy=True</code> you only load the table you've queried, so memory consumption in the pod dropped from 1.2Gb to 140Mb.</p>
