<p>We're moving all of our infrastructure to Google Kubernetes Engine (GKE) - we currently have 50+ AWS machines with lots of APIs, Services, Webapps, Database servers and more.</p>

<p>As we have already dockerized everything, it's time to start moving everything to GKE.</p>

<p>I have a question that may sound too basic, but I've been searching the Internet for a week and did not found any reasonable post about this</p>

<p>Straight to the point, which of the following approaches is better and why:</p>

<ol>
<li><p>Having multiple node pools with multiple machine types and always specify in which pool each deployment should be done; or</p></li>
<li><p>Having a single pool with lots of machines and let Kubernetes scheduler do the job without worrying about where my deployments will be done; or</p></li>
<li><p>Having BIG machines (in multiple zones to improve clusters' availability and resilience) and let Kubernetes deploy everything there.</p></li>
</ol>

<h2>List of consideration to be taken merely as hints, I do not pretend to describe best practice.</h2>

<ul>
<li><p>Each pod you add brings with it some <strong>overhead</strong>, but you increase in terms of flexibility and availability making failure and maintenance of nodes to be less impacting the production. </p></li>
<li><p>Nodes too small would cause a big waste of resources since sometimes will be not possible to schedule a pod even if the total amount of free RAM or CPU across the nodes would be enough, you can see this issue similar to memory <strong>fragmentation</strong>.</p></li>
<li><p>I guess that the sizes of PODs and their memory and CPU request are not similar, but I do not see this as a big issue in principle and a reason to go for 1). I do not see why a big POD should run merely on big machines and a small one should be scheduled on small nodes. <strong>I would rather use 1) if you need a different memoryGB/CPUcores ratio to support different workloads.</strong> </p></li>
</ul>

<p>I would advise you to run some test in the initial phase to understand which is the size of the biggest POD and the average size of the workload in order to properly chose the machine types. Consider that having 1 POD that exactly fit in one node and assign to it is not the right to proceed(virtual machine exist for this kind of scenario). Since fragmentation of resources would easily cause to impossibility to schedule a large node.</p>

<ul>
<li><p>Consider that their size will likely increase in the future and to <a href="https://stackoverflow.com/questions/45037213/how-to-vertically-scale-google-cloud-instance-without-stopping-running-app">scale vertically</a> is not always this immediate and you need to switch off machine and terminate pods, I would <strong>oversize a bit</strong> taking this issue into account and since scaling horizontally is way easier. </p></li>
<li><p>Talking about the machine type you can decide to go for a machine 5xsize the biggest POD you have (or 3x? or 10x?). <strong>Oversize a bit</strong> as well the numebr of nodes of the cluster to take into account overheads, fragmentation and in order to still have free resources.</p>

<ol>
<li><blockquote>
  <p>Remember that you have an hard limit of 100 pods each node and 5000 nodes.</p>
</blockquote></li>
<li><blockquote>
  <p><a href="https://cloud.google.com/compute/docs/networks-and-firewalls#egress_throughput_caps" rel="nofollow noreferrer">Remember</a> that in GCP the network egress throughput cap is dependent on the number of vCPUs that a virtual machine instance has. Each vCPU has a 2 Gbps egress cap for peak performance. However each additional vCPU increases the network cap, up to a theoretical maximum of 16 Gbps for each virtual machine.</p>
</blockquote></li>
<li><blockquote>
  <p>Regarding the prices of the virtual machines notice that there is no difference in price buying two machines with size x or one with size 2x. Avoid to customise the size of machines because rarely is convenient, if you feel like your workload needs more cpu or mem go for HighMem or HighCpu machine type. </p>
</blockquote></li>
</ol></li>
</ul>

<p>P.S. Since you are going to build a pretty big Cluster, check the <a href="https://kubernetes.io/docs/tasks/administer-cluster/dns-horizontal-autoscaling/" rel="nofollow noreferrer">size of the DNS</a> </p>

<p>I will add any consideration that it comes to my mind, consider in the future to update your question with the description of the path you chose and the issue you faced.</p>
