<p>I need a suggestion on managing thousands of services and statefulsets(1 pod) in a Kubernetes cluster. </p>

<p>Each of the pods needs at least 500mb of memory and these statefulsets are not always up, it will be down for some time &amp; run for some time. </p>

<p>What kind of nodes I should use and what kind of tools I should use here to reduce the billing?</p>

<p>Thank you</p>

<p>The type of nodes you need to use would be based on the type of workload your pods would be performing. <a href="https://cloud.google.com/compute/docs/machine-types" rel="nofollow noreferrer">https://cloud.google.com/compute/docs/machine-types</a> </p>

<p>You reduce the billing of the cluster by always maximizing resource utilization. </p>

<p><strong>Cluster Autoscaller</strong> will help you achieve the same. Increases /Decreases the number of nodes in the cluster based on load.
<a href="https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler" rel="nofollow noreferrer">https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler</a></p>

<p>You can also reduce your billing by using <strong>Preemptible VM's</strong> or <strong>Committed Use Discounts</strong></p>

<p><strong>Preemptible VM's</strong> Since all the nodes in the Kubernetes cluster can be replaced by a similar VM. (cattle not pets!) <a href="https://cloud.google.com/compute/docs/instances/preemptible" rel="nofollow noreferrer">https://cloud.google.com/compute/docs/instances/preemptible</a></p>

<p><strong>Committed Use Discounts</strong> GCP also reduces the cost of the VM's if you are committed to using the VM's for a longer duration
<a href="https://cloud.google.com/compute/docs/instances/signing-up-committed-use-discounts" rel="nofollow noreferrer">https://cloud.google.com/compute/docs/instances/signing-up-committed-use-discounts</a></p>
