<p>I have a sandbox Kubernetes cluster, in which I shutdown all pods at night, so it can scale down with the <code>cluster-autoscaler</code> add-on.</p>

<p>The problem is, it almost always keep the master plus 2 nodes running.</p>

<p>Looking into <code>cluster-autoscaler</code> logs, I see the problem seems to be this:</p>

<pre><code>Fast evaluation: node ip-172-16-38-51.ec2.internal cannot be removed: non-deamons set, non-mirrored, kube-system pod present: dns-controller-3586597043-531v5
Fast evaluation: node ip-172-16-49-207.ec2.internal cannot be removed: non-deamons set, non-mirrored, kube-system pod present: heapster-564189836-3h2ts
Fast evaluation: node ip-172-16-49-207.ec2.internal cannot be removed: non-deamons set, non-mirrored, kube-system pod present: kube-dns-1321724180-c0rjr
Fast evaluation: node ip-172-16-49-207.ec2.internal cannot be removed: non-deamons set, non-mirrored, kube-system pod present: kube-dns-autoscaler-265231812-dv17j
Fast evaluation: node ip-172-16-49-207.ec2.internal cannot be removed: non-deamons set, non-mirrored, kube-system pod present: kubernetes-dashboard-2396447444-6bwtq
Fast evaluation: node ip-172-16-49-207.ec2.internal cannot be removed: non-deamons set, non-mirrored, kube-system pod present: monitoring-influxdb-grafana-v4-50v9d
Fast evaluation: node ip-172-16-51-146.ec2.internal cannot be removed: non-deamons set, non-mirrored, kube-system pod present: cluster-autoscaler-776613730-kqgk2
</code></pre>

<p>and because those pods are spread, cluster-autoscaler ends up keeping 2 or more nodes up even when there is nothing running in the default namespace...</p>

<p>Is there a way of forcing or inducing Kubernetes to schedule all those pods together?</p>

<p>The idea is to make the cluster run at night with the master plus one node only. If there isn't, I was thinking on add a <code>Scheduled Action</code> to the <code>AutoScale Group</code>, so it would be forced to run all in the same node.</p>

<p>An alternative to using nodeselector would be to use <a href="https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#inter-pod-affinity-and-anti-affinity-beta-feature" rel="nofollow noreferrer">inter-pod affinity</a> to ensure that your pods get packed better. </p>

<p>'Preferred' rather than 'Required' affinity can be used to ensure that Kubernetes will try and schedule your pods together on the same node, but if it cannot, they will schedule on different nodes.</p>

<p>From the documentation, it will let you specify rules like:</p>

<blockquote>
  <p>This pod should (or, in the case of
  anti-affinity, should not) run in an X if that X is already running
  one or more pods that meet rule Y. X is a topology domain like node, rack, cloud provider zone, cloud provider region, etc.</p>
</blockquote>
