<p>I have a kubernetes cluster with 4 nodes. I have a pod deployed as a deployment, with 8 replicas. When I deployed this, kubernetes sometimes schedule 4 pods in node1, and the rest of the 4 pods in node2. In this case node3 and node4 don't have this container running (but other containers running there)</p>

<p>I do understand Pod affinity and <a href="https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity" rel="noreferrer">anti-affinity</a> , where they have the <a href="https://kubernetes.io/docs/tutorials/stateful-application/zookeeper/#tolerating-node-failure" rel="noreferrer">Zookeeper</a> example for pod-anti-affinity, which is great. This would make sure that no 2 pods would deploy on the same node. </p>

<p>This is fine, however my requirement is slightly different where I want to restrict the maximum number of the pods k8s can deploy to one node with node anti-affinity.</p>

<p>I need to make sure that not more than 3 instance of same pods are deployed on a node in my above example. I thought of setting a memory/cpu limit on pods but that seemed like a bad idea as I have nodes with different configuration. Is there any way to achieve this? </p>

<p>( Update 1 ) - I understand that my questions wasn't clear enough. To clarify further, what I want is to limit the instance of a pod to a maximum of 3 per node for a particular deployment. Example, how do I tell k8s to not deploy more than 3 instances of nginx pod per node? The restriction should only be applied to the nginx deployments and not other deployments.</p>

<p>( Update 2 ) - To further explain with a scenario. 
A k8s cluster, with 4 worker nodes. 
2 Deployments</p>

<ol>
<li>A nginx deployment -> replicas = 10</li>
<li>A custom user agent deployment -> Replicas 10</li>
</ol>

<p>Requirement - Hey kubernetes, I want to schedule 10 Pods of the "custom user agent" pod (Pod #2 in this example) in 4 nodes, but I want to make sure that each node may have only a maximum of 3 pods of the 'custom user agent'. For the 'nginx' pod, there shouldnt' be any such restriction, which means I don't mind if k8s schedule 5 nginx in one node and the rest of the 5 in the second node. </p>

zeroweb
<p>I myself didn't find official documentation for this. but i think you can use podantiaffinity with <code>preferredDuringSchedulingIgnoredDuringExecution</code> option. this will prevent k8s from placing the same pods on a single node, but if that is not possible it will select the most eligible existing node. official doc <a href="https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/" rel="nofollow noreferrer">here</a></p>
<pre><code>affinity:
    podAntiAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - podAffinityTerm:
          labelSelector:
            matchLabels:
              name: deployment-name
          topologyKey: kubernetes.io/hostname
        weight: 100
</code></pre>
