<p>What I'm trying to do is using Celery with Kubernetes. I'm using Redis as the message broker in a different pod and I have multiple pods for each queue of Celery.</p>

<p>Imagine if I have 3 queues, I would have 3 different pods (i.e workers) that can accept and handle the requests. </p>

<p>Everything is working fine so far but my question is, what would happen if I clone the pod of one of queues to have two pods for one single queue? </p>

<p>I think client (i.e Django) creates a new message using Redis to send to the worker and start the job but it's not clear to me what would happen because I have two pods listening to the same queue? Does the first pod accept the request and start the job and prevents the other pod to accept the request? </p>

<p><em>(I tried to search a bit on the documentation of Celery to see if I can find any clues but I couldn't. That's why I'm asking this question)</em></p>

<p>I guess you are using basic task type, which employs 'direct' queue type, not 'fanout' or 'topic' queue, the latter two have much difference, which will not be discussed here.</p>

<p>While using Redis as broker transport, celery/kombu use a Redis <code>list</code>  object as a storage of queue (<a href="https://github.com/celery/kombu/blob/master/kombu/transport/redis.py" rel="noreferrer">source</a>), use command <code>LPUSH</code> to publish message, <code>BRPOP</code> to consume the message.</p>

<p>In short, <code>BRPOP</code>(<a href="https://redis.io/commands/brpop" rel="noreferrer">doc</a>) blocks the connection when there are no elements to pop from the given lists, if the list is not empty, an element is popped from the tail of the given list. It is guaranteed that <a href="https://redis.io/commands/blpop#blocking-behavior" rel="noreferrer">this operation is atomic</a>, no two connection could get the same element. </p>

<p>Celery leverage this feature to guarantees <em>at-least-once</em> message delivery. use of acknowledgment doesn't affect this guarantee.</p>

<p>In your case, there are multiple celery workers across multiple pods, but all of them connected to one same Redis server, all of them blocked for the same key, try to pop an element from the same list object. when new message arrived, there will be one and only one worker could get that message.</p>

<hr>
