<p>I create a Pod with Replica count of say 2, which runs an application ( a simple web-server), basically it's always running command - However due to mis-configuration, sometimes the command exits and the pod is then Terminated.</p>

<p>Due to default <code>restartPolicy</code> of <code>Always</code> the pod (and hence the container) is restarted and eventually the Pod status is <code>CrashLoopBackOff</code>. </p>

<p>If I do a <code>kubectl describe deployment</code>, it shows Condition as <code>Progressing=True</code> and <code>Available=False</code>.</p>

<p>This looks fine - the question is - how do I mark my deployment as 'failed' in the above case? </p>

<p>Adding a <code>spec.ProgressDeadlineSeconds</code> doesn't seem to be having an effect. </p>

<p>Will simply saying <code>restartPolicy</code> as <code>Never</code> be enough in the Pod specification?</p>

<p>A related question, is there a way of getting this information as a trigger/webhook, without doing a <code>rollout status</code> watch? </p>

<h2>A bit of theory</h2>
<p>Regarding your question:</p>
<blockquote>
<p>How do I mark my deployment as 'failed' in the above case?</p>
</blockquote>
<p>Kubernetes gives you <a href="https://cloud.google.com/blog/products/gcp/kubernetes-best-practices-setting-up-health-checks-with-readiness-and-liveness-probes" rel="nofollow noreferrer">two types</a> of health checks:</p>
<p><strong>1 ) Readiness</strong><br>
Readiness probes are designed to let Kubernetes know when your app is <strong>ready to serve traffic</strong>. <br>
Kubernetes makes sure the readiness probe passes before allowing a service to send traffic to the pod. <br>
If a readiness probe starts to fail, Kubernetes stops sending traffic to the pod until it passes.</p>
<p><strong>2 ) Liveness</strong><br>
Liveness probes let Kubernetes know if your app is <strong>alive or dead</strong>. <br>
If you app is alive, then Kubernetes leaves it alone. If your app is dead, Kubernetes removes the Pod and starts a new one to replace it.</p>
<p>At the moment (v1.19.0) , Kubernetes has support for 3 types mechanisms for implementing liveness and readiness probes:</p>
<p><strong>A ) ExecAction</strong>: Executes a specified command inside the container. The diagnostic is considered successful if the command exits with a status code of 0.</p>
<p><strong>B ) TCPSocketAction</strong>: Performs a TCP check against the Pod's IP address on a specified port. The diagnostic is considered successful if the port is open.</p>
<p><strong>C ) HTTPGetAction</strong>: Performs an HTTP GET request against the Pod's IP address on a specified port and path. The diagnostic is considered successful if the response has a status code greater than or equal to 200 and less than 400.</p>
<hr />
<h2>In your case:<br></h2>
<p>If the process in your container is able to crash on its own whenever it encounters an issue or becomes unhealthy, <strong>you do not necessarily need a liveness probe</strong>; the kubelet will automatically perform the correct action in accordance with the Pod's restartPolicy.</p>
<p>I think that in your case (the need to refer to a deployment as succeed / failed and take the proper action) <strong>you should</strong>:</p>
<p><strong>Step 1:</strong> <br>
Setup a HTTP/TCP  <strong>readiness Probe</strong> - for example:</p>
<pre><code>   readinessProbe:
      httpGet:
         path: /health-check
         port: 8080
      initialDelaySeconds: 5
      periodSeconds: 5
      failureThreshold: 2
</code></pre>
<p>Where:</p>
<p><code>initialDelaySeconds</code> — The number of seconds since the container has started before the readiness probe can be initiated.</p>
<p><code>periodSeconds</code> — How often to perform the readiness probe.</p>
<p><code>failureThreshold</code> — The number of tries to perform the readiness probe if the probe fails on pod start.</p>
<p><strong>Step 2:</strong><br>
Choose the relevant <a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#strategy" rel="nofollow noreferrer">rolling update strategy</a> and how you should handle cases of failures of new pods (consider reading <a href="https://stackoverflow.com/questions/52121422/how-to-automatically-stop-rolling-update-when-crashloopbackoff/64451702#64451702">this</a> thread for examples).</p>
<p>A few references you can follow:</p>
<p><a href="https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#container-probes" rel="nofollow noreferrer">Container probes</a><br>
<a href="https://blog.colinbreck.com/kubernetes-liveness-and-readiness-probes-how-to-avoid-shooting-yourself-in-the-foot/#:%7E:text=Kubernetes%20uses%20liveness%20probes%20to%20know%20when%20to%20restart%20a%20container.&amp;text=Kubernetes%20uses%20readiness%20probes%20to,the%20backends%20for%20a%20service." rel="nofollow noreferrer">Kubernetes Liveness and Readiness Probes</a><br>
<a href="https://alibaba-cloud.medium.com/kubernetes-configure-liveness-and-readiness-probes-588aa2cab90d" rel="nofollow noreferrer">Kubernetes : Configure Liveness and Readiness Probes</a><br>
<a href="https://www.magalix.com/blog/kubernetes-and-containers-best-practices-health-probes" rel="nofollow noreferrer">Kubernetes and Containers Best Practices - Health Probes</a><br>
<a href="https://medium.com/kubernetes-tutorials/creating-liveness-probes-for-your-node-js-application-in-kubernetes-285fd6d22497" rel="nofollow noreferrer">Creating Liveness Probes for your Node.js application in Kubernetes</a></p>
<hr />
<h3>A Failed Deployment</h3>
<p>A deployment (or the rollout process) will be considered as <strong>Failed</strong>
if it tries to deploy its newest ReplicaSet without ever completing over and over again until the <code>progressDeadlineSeconds</code> interval has exceeded.</p>
<p>Then K8S you update the status with:</p>
<pre><code>Conditions:
  Type            Status  Reason
  ----            ------  ------
  Available       True    MinimumReplicasAvailable
  Progressing     False   ProgressDeadlineExceeded
  ReplicaFailure  True    FailedCreate
</code></pre>
<p>Read more in <a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#failed-deployment" rel="nofollow noreferrer">here</a>.</p>
