<p>I just switched from ForkPool to gevent with concurrency (5) as the pool method for Celery workers running in Kubernetes pods. After the switch I've been getting a non recoverable erro in the worker:</p>
<p><code>amqp.exceptions.PreconditionFailed: (0, 0): (406) PRECONDITION_FAILED - delivery acknowledgement on channel 1 timed out. Timeout value used: 1800000 ms. This timeout value can be configured, see consumers doc guide to learn more</code></p>
<p>The broker logs gives basically the same message:</p>
<p><code>2021-11-01 22:26:17.251 [warning] &lt;0.18574.1&gt; Consumer None4 on channel 1 has timed out waiting for delivery acknowledgement. Timeout used: 1800000 ms. This timeout value can be configured, see consumers doc guide to learn more</code></p>
<p>I have the <code>CELERY_ACK_LATE</code> set up, but was not familiar with the necessity to set a timeout for the acknowledgement period. And that never happened before using processes. Tasks can be fairly long (60-120 seconds sometimes), but I can't find a specific setting to allow that.</p>
<p>I've read in another post in other forum a user who set the timeout on the broker configuration to a huge number (like 24 hours), and was also having the same problem, so that makes me think there may be something else related to the issue.</p>
<p>Any ideas or suggestions on how to make worker more resilient?</p>

<p>The accepted answer is the correct answer. However, if you have an existing RabbitMQ server running and do not want to restart it, you can dynamically set the configuration value by running the following command on the RabbitMQ server:</p>
<p><code>rabbitmqctl eval 'application:set_env(rabbit, consumer_timeout, 36000000).'</code></p>
<p>This will set the new timeout to 10 hrs (36000000ms). For this to take effect, you need to restart your workers though. Existing worker connections will continue to use the old timeout.</p>
<p>You can check the current configured timeout value as well:</p>
<p><code>rabbitmqctl eval 'application:get_env(rabbit, consumer_timeout).'</code></p>
<p>If you are running RabbitMQ via Docker image, here's how to set the value: Simply add <code>-e RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS=&quot;-rabbit consumer_timeout 36000000&quot;</code> to your <code>docker run</code> OR set the environment <code>RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS</code> to <code>&quot;-rabbit consumer_timeout 36000000&quot;</code>.</p>
<p>Hope this helps!</p>
