<p>I have been deploying apps to Kubernetes for the last 2 years. And in my org, all our apps(especially stateless) are running in Kubernetes. I still have a fundamental question, just because very recently we found some issues with respect to our few python apps.</p>

<p>Initially when we deployed, our python apps(Written in Flask and Django), we ran it using <code>python app.py</code>. It's known that, because of GIL, python really doesn't have support for system threads, and it will only serve one request at a time, but in case the one request is CPU heavy, it will not be able to process further requests. This is causing sometimes the health API to not work. We have observed that, at this moment, if there is a single request which is not IO and doing some operation, we will hold the CPU and cannot process another request in parallel. And since it's only doing fewer operations, we have observed there is no increase in the CPU utilization also. This has an impact on how <code>HorizontalPodAutoscaler</code> works, its unable to scale the pods.</p>

<p>Because of this, we started using <code>uWSGI</code> in our pods. So basically <code>uWSGI</code> can run multiple pods under the hood and handle multiple requests in parallel, and automatically spin new processes on demand. But here comes another problem, that we have seen, <code>uwsgi</code> is lacking speed in auto-scaling the process tocorrected serve the request and its causing <code>HTTP 503</code> errors, Because of this we are unable to serve our few APIs in 100% availability. </p>

<p>At the same time our all other apps, written in <code>nodejs</code>, <code>java</code> and <code>golang</code>, is giving 100% availability.</p>

<p>I am looking at what is the best way by which I can run a python app in 100%(99.99) availability in Kubernetes, with the following</p>

<blockquote>
  <ol>
  <li>Having health API and liveness API served by the app</li>
  <li>An app running in Kubernetes</li>
  <li>If possible without uwsgi(Single process per pod is the fundamental docker concept)</li>
  <li>If with uwsgi, are there any specific config we can apply for k8s env</li>
  </ol>
</blockquote>

<p>We use Twisted's WSGI server with 30 threads and it's been solid for our Django application. Keeps to a single process per pod model which more closely matches Kubernetes' expectations, as you mentioned. Yes, the GIL means only one of those 30 threads can be running Python code at time, but as with most webapps, most of those threads are blocked on I/O (usually waiting for a response from the database) the vast majority of the time. Then run multiple replicas on top of that both for redundancy and to give you true concurrency at whatever level you need (we usually use 4-8 depending on the site traffic, some big ones are up to 16).</p>
