<p>We are migrating our infrastructure to kubernetes. I am talking about a part of it, that contains of an api for let's say customers (we have this case for many other resources). Let's consider we have a billion customers, each with some data etc. and we decided they deserve a specialized api just for them, with its own db, server, domain, etc.</p>
<p>Kubernetes has the notion of nodes and pods. So we said &quot;ok, we dedicate node X with all its resources to this particular api&quot;. And now the question:</p>
<p>Why would I used multiple pods each of them containing the same nginx + fpm and code, and limit it to a part of traffic and resources, and add an internal lb, autoscale, etc., instead of having a single pod, with all node resources?</p>
<p>Since each pod adds a bit of extra memory consumption this seems like a waste to me. The only upside being the fact that if something fails only part of it goes down (so maybe 2 pods would be optimal in this case?).</p>
<p>Obviously, would scale the nodes when needed.</p>
<p>Note: I'm not talking about a case where you have multiple pods with different stuff, I'm talking about that particular case.</p>
<p>Note 2: The db already is outside this node, on it's own pod.</p>
<p>Google fails me on this topic. I find hundreds of post with &quot;how to configure things, but 0 with WHY?&quot;.</p>

<blockquote>
<p>Why would I used multiple pods each of them containing the same nginx + fpm and code, and limit it to a part of traffic and resources, and add an internal lb, autoscale, etc., instead of having a single pod, with all node resources?</p>
</blockquote>
<blockquote>
<p>Since each pod adds a bit of extra memory consumption this seems like a waste to me. The only upside being the fact that if something fails only part of it goes down (so maybe 2 pods would be optimal in this case?).</p>
</blockquote>
<p>This comes down to the question, should I scale my app <strong>vertically (larger instance)</strong> or <strong>horizontally (more instances)</strong>.</p>
<p>First, try to avoid using only a <em>single instance</em> since you probably want more redundancy if you e.g. upgrade a Node. A single instance may be a good option if you are OK with some downtime sometimes.</p>
<h2>Scale app vertically</h2>
<p>To scale an app vertically, by changing the instance to a bigger, is a viable alternative that sometimes is a good option. Especially when the app can not be scaled horizontally, e.g. an app that use <em>leader election</em> pattern - typically listen to a specific event and react. There is however a limit by how much you can scale an app vertically.</p>
<h2>Scale app horizontally</h2>
<p>For a stateless app, it is usually <strong>much easier</strong> and cheaper to scale an app horizontally by adding more instances. You typically want more than one instance anyway, since you want to tolerate that a Node goes down for maintenance. This is also possible to do for a large scale app to very many instances - and the cost scales linearly. However, not every app can scale horizontally, e.g. a distributed database (replicated) can typically not scale well horizontally unless you shard the data. You can even use <a href="https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/" rel="nofollow noreferrer">Horizontal Pod Autoscaler</a> to automatically adjust the number of instances depending on how busy the app is.</p>
<h2>Trade offs</h2>
<p>As described above, horizontal scaling is usually easier and preferred. But there are trade offs - you would probably not want to run thousands of instances when you have low traffic - an instance has some resource overhead costs, also in maintainability. For <em>availability</em> you should run at least 2 pods and make sure that they does not run on the same node, if you have a <em>regional</em> cluster, you want to make sure that they does not run on the same <em>Availability Zone</em> - for availability reasons. Consider 2-3 pods when your traffic is low, and use <a href="https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/" rel="nofollow noreferrer">Horizontal Pod Autoscaler</a> to automatically scale up to more instance when you need. In the end, this is a number game - resources cost money - but you want to provide a good service for your customers as well.</p>
