<p>Got a simple REST API server built with python gunicorn, which runs multiple threads to accept requests. After running for some time, some of these threads crash. Got a script to detect the number of dead threads (using log files). Once this number crosses some threshold, we want to restart gunicorn. This script is configured to be used as <a href="https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/" rel="nofollow noreferrer">liveness probe</a>.</p>
<p>The script works fine and restarts the pod as expected. But there are a few live threads that are still processing requests. Also, gunicorn keeps a backlog queue of accepted requests that it cannot process yet, since other requests are processing. Is there a way to specify a delay for the pod restart so the other running threads and the backlog requests have some time to finish processing?</p>

arun
<p>You can use a prestop hook. Offcial docs <a href="https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks" rel="nofollow noreferrer">here</a></p>
<p>How to use documented <a href="https://kubernetes.io/docs/tasks/configure-pod-container/attach-handler-lifecycle-event/#define-poststart-and-prestop-handlers" rel="nofollow noreferrer">here</a>.</p>
<p>You can also use terminationGracePeriodSeconds to allow graceful termination of pod.</p>
<p>Best Practices <a href="https://cloud.google.com/blog/products/gcp/kubernetes-best-practices-terminating-with-grace" rel="nofollow noreferrer">here</a></p>
