<p>I am trying to figure out if there is a way to force a pod that is stuck on <code>containerCreating</code> state (for valid reasons like can't mount an inaccessible NFS, etc.) to move to a failed state after a specific amount of time.</p>

<p>I have Kubernetes <code>jobs</code> that I'm running through a Jenkins pipeline. I'm using the job state (<code>type: completed|failed</code>) to determine the outcome and then I gather the results of the jobs (<code>kubectl get pods</code> + <code>kubectl logs</code>). It works well as long as the pods go into a known failed state like <code>ContainerCannotRun</code> or <code>Backofflimit</code> and therefore the <code>job</code> state goes to <code>failed</code>.</p>

<p>Where the problem arises is when a pod goes into <code>containerCreating</code> state and stays that way. Then, the job state stays <code>active</code> and will never change. Is there a way, in the <code>job</code> manifest to put something to force a pod that's in <code>containerCreating</code> state to move to a failed state after a certain amount of time?</p>

<p>Example:
<strong>pod status</strong></p>

<pre><code>    - image: myimage
      imageID: ""
      lastState: {}
      name: primary
      ready: false
      restartCount: 0
      state:
        waiting:
          reason: ContainerCreating
    hostIP: x.y.z.y
    phase: Pending
    qosClass: BestEffort
    startTime: "2020-05-06T17:09:58Z"
</code></pre>

<p><strong>job status</strong></p>

<pre><code>    active: 1
    startTime: "2020-05-06T17:09:58Z"
</code></pre>

<p>Thanks for any input.</p>

lapfrank
<p>As documented <a href="https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/#job-termination-and-cleanup" rel="nofollow noreferrer">here</a> use <code>activeDeadlineSeconds</code> or <code>backoffLimit</code></p>

<p>The <code>activeDeadlineSeconds</code> applies to the duration of the job, no matter how many Pods are created. Once a Job reaches <code>activeDeadlineSeconds</code>, all of its running Pods are terminated and the Job status will become type: Failed with reason: DeadlineExceeded.</p>

<p>Once <code>backoffLimit</code> has been reached the Job will be marked as failed and any running Pods will be terminated.</p>

<p>Note that a Job’s <code>activeDeadlineSeconds</code> takes precedence over its <code>backoffLimit</code>. Therefore, a Job that is retrying one or more failed Pods will not deploy additional Pods once it reaches the time limit specified by <code>activeDeadlineSeconds</code>, even if the <code>backoffLimit</code> is not yet reached.</p>

<pre><code>apiVersion: batch/v1
kind: Job
metadata:
  name: pi-with-timeout
spec:
  backoffLimit: 5
  activeDeadlineSeconds: 100
  template:
    spec:
      containers:
      - name: pi
        image: perl
        command: ["perl",  "-Mbignum=bpi", "-wle", "print bpi(2000)"]
      restartPolicy: Never
</code></pre>
