<p>I have multiple Node.js apps / Services running on Google Kubernetes Engine (GKE), Actually 8 pods are running. I didnot set up resources limit when I created the pods so now I'm getting CPU Unscheduled error.  </p>

<p>I understand I have to set up resource limits. From what I know, 1 CPU / Node = 1000Mi ? My question is, </p>

<p>1) what's the ideal resource limit I should set up? Like the minimum? for a Pod that's rarely used, can I set up 20Mi? or 50Mi? </p>

<p>2) How many Pods are ideal to run on a single Kubernetes Node? Right now I have 2 Nodes set up which I want to reduce to 1. </p>

<p>3) what do people use in Production? and for development Cluster?</p>

<p>Here are my Nodes</p>

<p>Node 1:</p>

<pre><code>Namespace                  Name                                                    CPU Requests  CPU Limits  Memory Requests  Memory Limits
  ---------                  ----                                                    ------------  ----------  ---------------  -------------
  default                    express-gateway-58dff8647-f2kft                         100m (10%)    0 (0%)      0 (0%)           0 (0%)
  default                    openidconnect-57c48dc448-9jmbn                          100m (10%)    0 (0%)      0 (0%)           0 (0%)
  default                    web-78d87bdb6b-4ldsv                                    100m (10%)    0 (0%)      0 (0%)           0 (0%)
  kube-system                event-exporter-v0.1.9-5c8fb98cdb-tcd68                  0 (0%)        0 (0%)      0 (0%)           0 (0%)
  kube-system                fluentd-gcp-v2.0.17-mhpgb                               100m (10%)    0 (0%)      200Mi (7%)       300Mi (11%)
  kube-system                kube-dns-5df78f75cd-6hdfv                               260m (27%)    0 (0%)      110Mi (4%)       170Mi (6%)
  kube-system                kube-dns-autoscaler-69c5cbdcdd-2v2dj                    20m (2%)      0 (0%)      10Mi (0%)        0 (0%)
  kube-system                kube-proxy-gke-qp-cluster-default-pool-7b00cb40-6z79    100m (10%)    0 (0%)      0 (0%)           0 (0%)
  kube-system                kubernetes-dashboard-7b89cff8-9xnsm                     50m (5%)      100m (10%)  100Mi (3%)       300Mi (11%)
  kube-system                l7-default-backend-57856c5f55-k9wgh                     10m (1%)      10m (1%)    20Mi (0%)        20Mi (0%)
  kube-system                metrics-server-v0.2.1-7f8dd98c8f-5z5zd                  53m (5%)      148m (15%)  154Mi (5%)       404Mi (15%)
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  CPU Requests  CPU Limits  Memory Requests  Memory Limits
  ------------  ----------  ---------------  -------------
  893m (95%)    258m (27%)  594Mi (22%)      1194Mi (45%)
</code></pre>

<p>Node 2:</p>

<pre><code> Namespace                  Name                                                    CPU Requests  CPU Limits  Memory Requests  Memory Limits
  ---------                  ----                                                    ------------  ----------  ---------------  -------------
  default                    kube-healthcheck-55bf58578d-p2tn6                       100m (10%)    0 (0%)      0 (0%)           0 (0%)
  default                    pubsub-function-675585cfbf-2qgmh                        100m (10%)    0 (0%)      0 (0%)           0 (0%)
  default                    servicing-84787cfc75-kdbzf                              100m (10%)    0 (0%)      0 (0%)           0 (0%)
  kube-system                fluentd-gcp-v2.0.17-ptnlg                               100m (10%)    0 (0%)      200Mi (7%)       300Mi (11%)
  kube-system                heapster-v1.5.2-7dbb64c4f9-bpc48                        138m (14%)    138m (14%)  301656Ki (11%)   301656Ki (11%)
  kube-system                kube-dns-5df78f75cd-89c5b                               260m (27%)    0 (0%)      110Mi (4%)       170Mi (6%)
  kube-system                kube-proxy-gke-qp-cluster-default-pool-7b00cb40-9n92    100m (10%)    0 (0%)      0 (0%)           0 (0%)
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  CPU Requests  CPU Limits  Memory Requests  Memory Limits
  ------------  ----------  ---------------  -------------
  898m (95%)    138m (14%)  619096Ki (22%)   782936Ki (28%)
</code></pre>

<p>My plan is to move all this into 1 Node. </p>

<p>According to kubernetes official documentation</p>

<p>1) You can go low in terms of memory and CPU, but you need to give enough CPU and memory to pods to function properly. I have gone as low as to CPU 100 and Memory 200 (It is highly dependent on the application you're running also the number of replicas)</p>

<p>2) <code>There should not be 100 pods per node</code> (This is the extreme case)</p>

<p>3) Production cluster are not of single node in any case. This is a very good read around <a href="https://techbeacon.com/one-year-using-kubernetes-production-lessons-learned" rel="nofollow noreferrer">kubernetes in production</a></p>

<p>But keep in mind, if you increase the number of pod on single node, you might need to increase the size (<code>in terms of resources</code>) of node.</p>

<p>Memory and CPU usage tends to grow proportionally with size/load on cluster</p>

<p>Here is the official documentation stating the requirements</p>

<blockquote>
  <p><a href="https://kubernetes.io/docs/setup/cluster-large/" rel="nofollow noreferrer">https://kubernetes.io/docs/setup/cluster-large/</a></p>
</blockquote>
