<p>Can anyone explain to me why when running my load test on one pod it gives better TPS rather than when scaling to two pods.</p>
<p>I expected that when running the same scenario with the same configuration on 2 pods the TPS will be increased but this is not what happened.</p>
<p>Is this normal behaviour that scaling horizontal not improve the total number of requests?</p>
<p>Please note that I didn't get any failures on one pod just scaled to 2 for high availability.</p>

<p>If you are using any sort of database, this is the place to optimize to increase TPS. Here is why:</p>
<p>Assume your database is running as fast as possible - The pod can handle the network connections and query against the database, but the database is slow because the CPU/Memory/TPS are already maxed; increasing the amount of TPS from pod 1 will not increase your TPS because the DB TPS is the limiting factor.</p>
<p>Now if you add pod 2, your DB still already has maxed out CPU/Memory/TPS, but now it has to use some of the CPU it was using before to complete transactions, to manage DB connections from pod 2 that have to be queued until the CPU/Memory of the DB can handle them; ultimately lowering TPS of the DB and TPS of the entire app.</p>
<p>TLDR: If your DB is maxed, adding pods to make transactions against it lowers the TPS because the DB has to use resources that were actively handling transactions (your limiting factor for TPS) to handle queued DB connections instead.</p>
<p>To fix this, vertically scale your write DB's, horizontally scale your read DB's, optimize DB transactions with index's or better queries, use PGBouncer to manage DB connections, and make sure your DB transaction type is the most efficient for your use case.</p>
