<p>I have a kubernetes cluster with a few different pod types. </p>

<ul>
<li>An Nginx frontend, </li>
<li>A flask backend on gunicorn, </li>
<li>Redis, and </li>
<li>A Redis queue (RQ).</li>
</ul>

<p>Based on what I can tell, the default liveness probing for the frontend, and flask backend are sufficient (200 OK returning, as I have created a '/' backend that returns 200 and all my tasks should run quickly). Crash detection works well. </p>

<p>Additionally, I have a setup aliveness monitor that pings Redis with the Redis-cli. That also is working well.</p>

<p>However, I am not sure if the default configuration for the RQ is good enough. The pod has restarted itself a few times and is generally well behaved, but since I don't know the mechanism that is used, I'm worried.</p>

<p>My questions are: what is the liveness probe used by something like an RQ worker and what might I do to make sure it's robust?</p>

<p>Should I be using something like Supervisor or systemd? Any recommendations on which one?</p>

<p>It would appear that RQ sets a heartbeat key in Redis: <a href="https://github.com/rq/rq/blob/e43bce4467c3e1800d75d9cedf75ab6e7e01fe8c/rq/worker.py#L545-L561" rel="nofollow noreferrer">https://github.com/rq/rq/blob/e43bce4467c3e1800d75d9cedf75ab6e7e01fe8c/rq/worker.py#L545-L561</a></p>

<p>You could check if this exists somehow. This would probably require an exec probe though, and at this time I wouldn't recommend that as exec probes have several open bugs that cause zombie processes leading to escalating resource usage over time.</p>
