<p>Batch computations, Monte Carlo, using Docker image, multiple jobs running on Google cloud and managed by Kubernetes. No Replication Controllers, just multiple pods with NoRestart policy delivering computed payloads to our server. So far so good. Problem is, I have cluster with <code>N</code> nodes/minions, and have <code>M</code> jobs to compute, where <code>M &gt; N</code>. So I would like to fire <code>M</code> pods at once and tell Kubernetes to schedule it in such a way so that only <code>N</code> are running at a given time, and everything else is kept in Pending state. As soon as one pod is done, next is scheduled to run moving from Pending to Running and so on and so forth till all <code>M</code> pods are done.</p>

<p>Is it possible to do so? </p>

<p>Yes, you can have them all ask for a resource of which there's only one on each node, then the scheduler won't be able to schedule more than N at a time. The most common way to do this is to have each pod ask for a <a href="https://github.com/kubernetes/kubernetes/blob/9030f16071f4f6d46f0482995f0db5eaf96b9294/pkg/api/v1/types.go#L802" rel="nofollow"><code>hostPort</code></a> in the <code>ports</code> section of its <code>containers</code> spec.</p>

<p>However, I can't say I'm completely sure why you would want to limit the system to one such pod per node. If there are enough resources available to run multiple at a time on each node, it should speed up your job to let them run.</p>
