<p>With my team, we're currently building an API using FastAPI and we're really struggling to get good performances out of it once deployed to Kubernetes. We're using async calls as much as possible but overall sitting at ~8RPS / pod to stay under our SLA of P99 200ms.</p>
<p>For resources, we assign the following:</p>
<pre><code>resources:
    limits:
        cpu: 1
        memory: 800Mi
    requests:
        cpu: 600m
        memory: 100Mi
</code></pre>
<p>Surprisingly, such performance drops don't occur when running load tests on the API running locally in a Docker container. There we easily get ~200RPS on a single container with 120ms latency at P99...</p>
<p>Would anyone have an idea of what could go wrong in there and where I could start looking to find the bottleneck?</p>
<p>Cheers!</p>

dernat71
<p>First, try to request at least 1 CPU for your API, because if there are no available CPUs on the node, the pod will only use the reserved amount of CPUs which is 600m, so if you have another application with requests cpu=400m for example, kubernetes will run both applications on the same cpu, with 60% of the time for the API and 40% for the second application. While docker uses 1 CPU (maybe more) in localhost.</p>
<p>If you are using Uvicorn with multiple workers, you can also increase CPU limits to  or at least 2.</p>
<pre class="lang-yaml prettyprint-override"><code>   Resources:
     limits:
         processor: 2
         memory: 800Mi
     requests:
         processor: 1
         memory: 100Mi
</code></pre>
<p>Finally, there is a difference between your local machine CPUs and kubernetes cluster CPUs, if you want to get good performance, you can test  better CPUs and choose the most suitable one in terms of cost.</p>
