<p>I have a Flask-based app that is used as a front (HTTP API only) for an image processing task (face detection along with some clustering). The app is deployed to Kubernetes clusters and, unfortunately during load testing it dies.</p>
<p>The problem is that all Flask threads are reserved for request processing and the application can't reply to Kubernetes liveness probe (<code>/health</code> endpoint) via HTTP - so the whole pod gets restarted.</p>
<p>How can I resolve it? I thought about a grep-based liveness problem however it doesn't solve the problem. Another idea is to use celery, however, if Flask doesn't support async-processing I'll need to call <code>wait()</code> on a celery task which gives me exactly to the same place.</p>
<p>For now I don't consider returning <code>202</code> response along with URL for process monitoring.</p>
<p>Any other ideas?</p>

Opal
<p>How did you deploy Gunicorn etc?</p>
<p>FastAPI might be better suited for your use case, but migration might be prohibitive. It has built in async support which would should help you to scale better. I like tiangolo's docker containers for this.</p>
<p>How long does your image recognition take (seconds, milliseconds)?</p>
<p>If you must stick to your current design:</p>
<ol>
<li>increase timeout, but be aware that your customers have the same problem - they might time out.</li>
<li>Increase resources: More pods so that no pod has no resources left.</li>
</ol>
<p>If you're using Flask, be aware that the dev server is not meant for production deployment. Although it is multithreaded, it's not particularly performant, stable, or secure. Use a production WSGI server to serve the Flask application, such as Gunicorn, mod_wsgi, or something else.</p>
