<p>I have a batch job which breaks down in 3 tasks that each depend on the previous finishing before they can start:</p>
<ol>
<li>Run a single pod</li>
<li>Run N pods in parallel (<code>.spec.completions</code> = <code>.spec.parallelism</code> = N)</li>
<li>Run M pods in parallel (<code>.spec.completions</code> = <code>.spec.parallelism</code> = M)</li>
</ol>
<p>Each task has different resource requirements (CPU/MEM/STORAGE). Currently, I start job #1, when it finishes it runs a <code>kubectl</code> command to start job #2, and so on to job #3. I have 3 separate jobs.</p>
<p>Can I define a single job for these 3 tasks?</p>
<p>Maybe something like this:</p>
<ol>
<li>Run single pod for task #1</li>
<li>Define init container on task #2 to wait for task #1 to finish</li>
<li>Run N pods for task #2 using <code>.spec.completions</code></li>
<li>Define init container on task # to wait for task #2 to finish</li>
<li>Run M pods for task #3 using a different <code>.spec.completions</code> appropriate for task #3</li>
</ol>
<p>It's not clear to me if I can define separate <code>.spec.parallelism</code> and <code>.spec.completions</code> for different pods under the same job. And if I can define separate init containers to delay the start of the later tasks.</p>
<p>This may all require a more complete workflow engine like Argo (which we don't yet have available).</p>

David Parks
<p>I just figured out a way to do this.</p>
<p>The yaml file supports multiple files. So you can append multiple job definitions under a single yaml file, using <code>---</code> as the deliminator.</p>
<p>Here is an example:</p>
<pre><code>apiVersion: batch/v1
kind: Job
metadata:
  name: ge-test-job
spec:
  template:
    spec:
      containers:
      - name: ge-test-1
        image: improbableailab/model-free
        command: [&quot;perl&quot;, &quot;-Mbignum=bpi&quot;, &quot;-wle&quot;, &quot;print 1&quot;]
        resources:
          limits:
            memory: 200Mi
            cpu: 1000m
          requests:
            memory: 50Mi
            cpu: 500m
        volumeMounts:
        - mountPath: /jaynes-mounts
          name: ge-pvc
      restartPolicy: Never
      volumes:
        - name: ge-pvc
          persistentVolumeClaim:
            claimName: ge-pvc
  backoffLimit: 4
  ttlSecondsAfterFinished: 10
---
apiVersion: batch/v1
kind: Job
metadata:
  name: ge-test-job-2
spec:
  template:
    spec:
      containers:
        - name: ge-test-2
          image: improbableailab/model-free
          command: [&quot;perl&quot;, &quot;-Mbignum=bpi&quot;, &quot;-wle&quot;, &quot;print 1&quot;]
          resources:
            limits:
              memory: 200Mi
              cpu: 1000m
            requests:
              memory: 50Mi
              cpu: 500m
          volumeMounts:
            - mountPath: /jaynes-mounts
              name: ge-pvc
      restartPolicy: Never
      volumes:
        - name: ge-pvc
          persistentVolumeClaim:
            claimName: ge-pvc
  backoffLimit: 4
  ttlSecondsAfterFinished: 10
</code></pre>
<p>Now if you run</p>
<pre class="lang-sh prettyprint-override"><code>❯ kubectl apply -f job.yaml
job.batch/ge-test-job created
job.batch/ge-test-job-2 created
</code></pre>
