<p>Currently, I have 2 nodes on AWS</p>
<pre><code>NAME                                           STATUS   ROLES    AGE   VERSION
NODE-1-xxx-xxx-xxx-xxx.cn-north-1.compute.internal   Ready    &lt;none&gt;   15d   v1.16.13-eks-2ba888
NODE-2-xxx-xxx-xxx-xxx.cn-north-1.compute.internal   Ready    &lt;none&gt;   13d   v1.16.13-eks-2ba888
</code></pre>
<p>Here is also a screenshot of my CPU loads</p>
<p><strong>NODE 1</strong>
<a href="https://i.stack.imgur.com/vHJM2.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/vHJM2.png" alt="NODE !" /></a></p>
<p><strong>NODE 2</strong>
<a href="https://i.stack.imgur.com/zgGut.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/zgGut.png" alt="NODE 2" /></a></p>
<p>My problem is whenever I deploy my application to production I will max out my cpu usage on <strong>NODE 2</strong> and it will slow down the entire site</p>
<p>Here is my deployment config</p>
<pre><code>apiVersion: apps/v1
kind: Deployment
metadata:
  name: backend # name of the deployment
  namespace: backend
  labels: # these labels apply to the deployment
    app: root
    component: backend

spec:
  replicas: 2
  minReadySeconds: 20
  selector:
    matchLabels:
      app: root
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  template:
    metadata:
      labels: # these labels apply to our container
        app: root
        component: backend
        version: xxx_${BITBUCKET_BUILD_NUMBER}
    spec:
      containers:
      - name: backend # name of our container
        image: xxx/xxx_main:${BITBUCKET_BUILD_NUMBER} # the URI that we got from ECR
        imagePullPolicy: Always
        envFrom:
        - configMapRef:
            name: env
        ports:
        - containerPort: 3000 # expose the running contianer on port 3000
          name: backend
          protocol: TCP
        readinessProbe:
          tcpSocket:
            port: backend
          initialDelaySeconds: 20
          periodSeconds: 5
          timeoutSeconds: 1
          successThreshold: 1
          failureThreshold: 20

      imagePullSecrets:
       - name: xxx
</code></pre>
<p>Am I not scaling things out properly here? what is the point of having two nodes if only one is ever in use? How can i properly scale my applications to use multiple nodes?</p>

NooBskie
<p>This happens because the node scheduling algorithm is based on priority score with different priority algorithms contributing to the score. One such priority algorithm is the <code>ImageLocalityPriority</code> which adds a positive priority score for nodes already having the images used by the pod. So initially, a node that already has the first replica of the pod running, gets a small priority bump due to the <code>ImageLocalityPriority</code>. Once more and more replicas are added, the number of pods running on each node even out because other priorities like <code>BalancedResourceAllocation</code> etc also take affect.</p>
<p>There is also a <code>SelectorSpreadPriority</code> which helps in minimising
the number of pods belonging to a same service on a node. So if you create your <code>service</code> object before creating the deployment, it might help.</p>
<p>To enforce the pods to spread out, you should add <a href="https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#inter-pod-affinity-and-anti-affinity" rel="nofollow noreferrer">inter-pod anti-affinity constraints</a> to your pods.</p>
<p>You should also consider adding <a href="https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#requests-and-limits" rel="nofollow noreferrer">requests and limits</a> to your containers. This helps in spreading out the pods as the <code>LeastRequestedPriority</code> priority also kicks in. (<strong>Note:</strong> There is also <code>MostRequestedPriority</code> which adds priority for used nodes but it is not enabled by default).</p>
