<p>I am trying to deploy a simple zookeeper ensemble following the tutorial from the <a href="https://kubernetes.io/docs/tutorials/stateful-application/zookeeper/" rel="nofollow noreferrer">official Kubernetes website</a>. The tutorial states that I need</p>
<blockquote>
<p>a cluster with at least four nodes and each node requires at least 2
CPUs and 4 GiB of memory.</p>
</blockquote>
<p>I ignored this fact and created a cluster with 3 nodes of n1-standard-1 (1 vCPU, 3.73 GB Memory)
When I tried to apply .yaml file</p>
<pre><code>apiVersion: v1
kind: Service
metadata:
  name: zk-hs
  labels:
    app: zk
spec:
  ports:
    - port: 2888
      name: server
    - port: 3888
      name: leader-election
  clusterIP: None
  selector:
    app: zk
---
apiVersion: v1
kind: Service
metadata:
  name: zk-cs
  labels:
    app: zk
spec:
  ports:
    - port: 2181
      name: client
  selector:
    app: zk
---
apiVersion: policy/v1beta1
kind: PodDisruptionBudget
metadata:
  name: zk-pdb
spec:
  selector:
    matchLabels:
      app: zk
  maxUnavailable: 1
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: zk
spec:
  selector:
    matchLabels:
      app: zk
  serviceName: zk-hs
  replicas: 3
  updateStrategy:
    type: RollingUpdate
  podManagementPolicy: OrderedReady
  template:
    metadata:
      labels:
        app: zk
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                  - key: &quot;app&quot;
                    operator: In
                    values:
                      - zk
              topologyKey: &quot;kubernetes.io/hostname&quot;
      containers:
        - name: kubernetes-zookeeper
          imagePullPolicy: Always
          image: &quot;k8s.gcr.io/kubernetes-zookeeper:1.0-3.4.10&quot;
          resources:
            requests:
              memory: &quot;1Gi&quot;
              cpu: &quot;0.5&quot;
          ports:
            - containerPort: 2181
              name: client
            - containerPort: 2888
              name: server
            - containerPort: 3888
              name: leader-election
          command:
            - sh
            - -c
            - &quot;start-zookeeper \
          --servers=3 \
          --data_dir=/var/lib/zookeeper/data \
          --data_log_dir=/var/lib/zookeeper/data/log \
          --conf_dir=/opt/zookeeper/conf \
          --client_port=2181 \
          --election_port=3888 \
          --server_port=2888 \
          --tick_time=2000 \
          --init_limit=10 \
          --sync_limit=5 \
          --heap=512M \
          --max_client_cnxns=60 \
          --snap_retain_count=3 \
          --purge_interval=12 \
          --max_session_timeout=40000 \
          --min_session_timeout=4000 \
          --log_level=INFO&quot;
          readinessProbe:
            exec:
              command:
                - sh
                - -c
                - &quot;zookeeper-ready 2181&quot;
            initialDelaySeconds: 10
            timeoutSeconds: 5
          livenessProbe:
            exec:
              command:
                - sh
                - -c
                - &quot;zookeeper-ready 2181&quot;
            initialDelaySeconds: 10
            timeoutSeconds: 5
          volumeMounts:
            - name: datadir
              mountPath: /var/lib/zookeeper
      securityContext:
        runAsUser: 1000
        fsGroup: 1000
  volumeClaimTemplates:
    - metadata:
        name: datadir
      spec:
        accessModes: [ &quot;ReadWriteOnce&quot; ]
        resources:
          requests:
            storage: 10Gi
</code></pre>
<p>And of course, I got the error <strong>PodUnschedulable</strong></p>
<p>In this file, I cloud not find anything that says that I need a cluster of 4 nodes and with 2 CPUs and 4G Ram.
So:</p>
<ul>
<li>What defines how much of resources do my deployment needs?</li>
<li>How to understand/calculate the required resources of applications and their respective deployments beforehand?</li>
<li>Zookeeper runs on 2GB RAM according to requirements but this is only a recommended configuration.</li>
</ul>

<p>By default, a kubernetes nodes will <strong>not come empty.</strong> Instead, it has running processes before even running your apps workload:</p>
<ul>
<li>kubelet is running ( in each node)</li>
<li>kube-proxy is running as daemonset ( in each node)</li>
<li>container-runtime (Docker) is running in each node</li>
<li>other daemonset can be running (like aws-node DS in the case of EKS..).</li>
</ul>
<blockquote>
<p>We are here discussing Worker Nodes not Masters.</p>
</blockquote>
<p>So imagine all that, you will end up by choosing a respectful resources for each Node.</p>
<p>Not all Nodes must be with the same size. However you decide which size you need according to the type of your apps :</p>
<ul>
<li><p>If your apps eats memory more than CPUs (Like Java Apps), you will need to choose Node of <strong>[2CPU, 8GB]</strong> is better than <strong>[4CPUs, 8GB]</strong>.</p>
</li>
<li><p>If your apps eats CPUs more than memory (Like ML workload), you will need to choose the opposite; computing-optimized instances.</p>
</li>
<li><p>The golden rule 🏆 is to calculate the <strong>whole capacity</strong> is better than looking into the individual capacity for each  node.</p>
</li>
</ul>
<p>This means <strong>3 large</strong> nodes might be better than <strong>4 medium</strong> nodes in term of cost but also in term of the best usage of capacity.</p>
<p><a href="https://i.stack.imgur.com/le3cH.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/le3cH.png" alt="enter image description here" /></a></p>
<p>As conclusion, Node resource must be:</p>
<ul>
<li>no less than 2 CPUs</li>
<li>no less than 4GB memory</li>
</ul>
<p>Otherwise, you should expect capacity issues.</p>
<hr />
<p>Now, we reach the half of the answer: Identify the capacity of the cluster.</p>
<p>The second half is about answering <strong>how to assign resources for the each app (pod)</strong>.</p>
<p>This is fall into another question; How much your app consume ?</p>
<p>To answer this question, you need to monitor your app with APM tools like Prometheus + Grafana.</p>
<p>Once you get insight about the avergae of consumption, it's the time to set <strong>resources limits</strong> for your app (its pods).</p>
<p><strong>Limits</strong> might throttle the app, that's why, you need to set up alongside other things for horizontal auto-scaling:</p>
<ul>
<li>Run Pods inside Deployment to manage replicas and deployment.</li>
<li>HPA or  Horizontal Pod autoscaler: which monitors the pods of the deployment , then scale out/in according to thresholds (CPU, memory)</li>
</ul>
<p>As conclusion for this part, we can say:</p>
<p><strong>- Measure :</strong> start measure to identify <code>resources.limits</code> and <code>resources.requests</code>.</p>
<p><strong>- Measure:</strong> measure after running the app to identify again the needed resources.</p>
<p><strong>- Measure:</strong> Keep measure</p>
