<p>I have got the following template for a job:</p>
<pre><code>apiVersion: batch/v1
kind: Job
metadata:
  name: &quot;gpujob&quot;
spec:
  completions: 1
  backoffLimit: 0
  ttlSecondsAfterFinished: 600000
  template:
    metadata:
      name: batch
    spec:
      volumes:
        - name: data
          persistentVolumeClaim:
            claimName: &quot;test&quot;
      containers:
        - name: myhub
          image: smat-jupyterlab
          env:
            - name: JUPYTERHUB_COOKIE_SECRET
              value: &quot;sdadasdasda&quot;
          resources:
            requests:
              memory: 500Gi
            limits:
              nvidia.com/gpu: 1
          command: [&quot;/bin/bash&quot;, &quot;/usr/local/bin/jobscript.sh&quot;, smat-job]
          volumeMounts:
            - name: data
              mountPath: /data
      restartPolicy: Never
      nodeSelector:
        dso-node-role: &quot;inference&quot;
</code></pre>
<p>As you can see, I claim a lot of memory for the job. My Question is: Does the failed pod free the claimed resources, as soon as it is on a failed state? Due to regulations, I have to keep pods for one week in the cluster, otherwise I would just set a very low <code>ttlSecondsAfterFinished</code>. I read a lot of contradicting stuff in articles, but found nothing in the official docs.</p>
<p><a href="https://i.stack.imgur.com/mxRRB.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/mxRRB.png" alt="enter image description here" /></a></p>
<p><strong>TDLR: Does a failed Pod free claimed resources of a cluster? If no, what is a good way to do it?</strong></p>

<p>Yes, a failed or completed job will produce a container in <a href="https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#container-state-terminated" rel="nofollow noreferrer">Terminated</a> state, and therefore the resources allocated to it are freed.</p>
<p>You can easily confirm this by using the command:</p>
<pre><code>kubectl top pod
</code></pre>
<p>You should not see any pod associated with the failed job consuming resources.</p>
