<p dir="auto">Hi,<br>
I am having some issues running latest celery 3.0.6 with django, from time to time some unrecoverable error occours and the worker crashes.<br>
I am running with redis as backend, this is the packages I am using:</p>
<p dir="auto">django==1.3<br>
celery==3.0.6<br>
django-celery==3.0.6<br>
redis==2.4.12</p>
<p dir="auto">Here some settings:<br>
CELERY_SEND_EVENTS = False<br>
CELERY_IGNORE_RESULT = True<br>
CELERY_TASK_PUBLISH_RETRY = True<br>
CELERY_DISABLE_RATE_LIMITS = True<br>
CELERY_ACKS_LATE = False<br>
CELERY_MESSAGE_COMPRESSION = 'gzip'</p>
<p dir="auto">python 2.7.3</p>
<p dir="auto">and here is the error:</p>
<p dir="auto">Unrecoverable error: 'ValueError('could not convert string to float: {"body": "gAJ9cQEoVQdyZXRyaWVzcQJLAFUEdGFza3EDVRVlbnRpdHkudGFza3MuYWRkX2xvdmVxBFUIZXJyYmFja3NxBU5VCWNhbGxiYWNrc3EGTlUGa3dhcmdzcQd9cQgoVQllbnRpdHlfaWRxCUp/KmQAVQd1c2VyX2lkcQpKjyENAFUPcG9wdWxhcl9mYXNoaW',)'</p>
<p dir="auto">and the stacktrace:</p>
<p dir="auto">Stacktrace (most recent call last):</p>
<p dir="auto">File "celery/worker/<strong>init</strong>.py", line 353, in start<br>
component.start()</p>
<p dir="auto">File "celery/worker/consumer.py", line 369, in start<br>
self.consume_messages()</p>
<p dir="auto">File "celery/worker/consumer.py", line 449, in consume_messages<br>
on_poll_empty()</p>
<p dir="auto">File "kombu/transport/redis.py", line 637, in on_poll_empty<br>
self.cycle.on_poll_empty()</p>
<p dir="auto">File "kombu/transport/redis.py", line 209, in on_poll_empty<br>
channel.qos.restore_visible()</p>
<p dir="auto">File "kombu/transport/redis.py", line 102, in restore_visible<br>
start=start, num=num, withscores=True)</p>
<p dir="auto">File "redis/client.py", line 1076, in zrevrangebyscore<br>
return self.execute_command(_pieces, *_options)</p>
<p dir="auto">File "kombu/transport/redis.py", line 546, in execute_command<br>
return self.parse_response(conn, command_name, **options)</p>
<p dir="auto">File "redis/client.py", line 347, in parse_response<br>
return self.response_callbacks[command_name](response, **options)</p>
<p dir="auto">File "redis/client.py", line 127, in zset_score_pairs<br>
return list(izip(it, imap(score_cast_func, it)))</p>
<p dir="auto">Let me know if I missed some info about this and if I need to be more specific about my current setup/environment.</p>
<p dir="auto">ps.<br>
Since I was running an old python redis client (2.4.x) I tried to go for the 2.6.2 but that did not helped.</p>
<p dir="auto">I'm running django-celery 3.0.6 on Heroku with a Redis broker.  About 1 minute after starting, Celery crashes with this traceback:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="2012-08-24T01:34:25+00:00 app[celeryd.1]: [2012-08-23 21:34:25,498: ERROR/MainProcess] Unrecoverable error: ValueError('could not convert str
ing to float: {&quot;body&quot;: &quot;gAJ9cQEoVQdyZXRyaWVzcQJLAFUEdGFza3EDVSZmcHMudGFza3MuY2hlY2tfcGF5bWVudF9zdGF0dXNfY2xlYW51cHEEVQhlcnJiYWNrc3EFTlUJY2Fsb
GJhY2tzcQZOVQZrd2FyZ3NxB31xCFUDZXRhcQlOVQRhcmdzcQpdcQtVAmlkcQxVJDBlMDVhYj',)
2012-08-24T01:34:25+00:00 app[celeryd.1]: Traceback (most recent call last):
2012-08-24T01:34:25+00:00 app[celeryd.1]:   File &quot;/app/lib/python2.7/site-packages/celery/worker/__init__.py&quot;, line 353, in start
2012-08-24T01:34:25+00:00 app[celeryd.1]:     component.start()
2012-08-24T01:34:25+00:00 app[celeryd.1]:   File &quot;/app/lib/python2.7/site-packages/celery/worker/consumer.py&quot;, line 369, in start
2012-08-24T01:34:25+00:00 app[celeryd.1]:     self.consume_messages()
2012-08-24T01:34:25+00:00 app[celeryd.1]:   File &quot;/app/lib/python2.7/site-packages/celery/worker/consumer.py&quot;, line 449, in consume_messages
2012-08-24T01:34:25+00:00 app[celeryd.1]:     on_poll_empty()
2012-08-24T01:34:25+00:00 app[celeryd.1]:   File &quot;/app/lib/python2.7/site-packages/kombu/transport/redis.py&quot;, line 637, in on_poll_empty
2012-08-24T01:34:25+00:00 app[celeryd.1]:     self.cycle.on_poll_empty()
2012-08-24T01:34:25+00:00 app[celeryd.1]:   File &quot;/app/lib/python2.7/site-packages/kombu/transport/redis.py&quot;, line 209, in on_poll_empty
2012-08-24T01:34:25+00:00 app[celeryd.1]:     channel.qos.restore_visible()
2012-08-24T01:34:25+00:00 app[celeryd.1]:   File &quot;/app/lib/python2.7/site-packages/kombu/transport/redis.py&quot;, line 102, in restore_visible
2012-08-24T01:34:25+00:00 app[celeryd.1]:     start=start, num=num, withscores=True)
2012-08-24T01:34:25+00:00 app[celeryd.1]:   File &quot;/app/lib/python2.7/site-packages/newrelic-1.4.0.137/newrelic/api/object_wrapper.py&quot;, line 1
66, in __call__
2012-08-24T01:34:25+00:00 app[celeryd.1]:     self._nr_instance, args, kwargs)
2012-08-24T01:34:25+00:00 app[celeryd.1]:   File &quot;/app/lib/python2.7/site-packages/newrelic-1.4.0.137/newrelic/api/function_trace.py&quot;, line 7
6, in literal_wrapper
2012-08-24T01:34:25+00:00 app[celeryd.1]:     return wrapped(*args, **kwargs)
2012-08-24T01:34:25+00:00 app[celeryd.1]:   File &quot;/app/lib/python2.7/site-packages/redis/client.py&quot;, line 1076, in zrevrangebyscore
2012-08-24T01:34:25+00:00 app[celeryd.1]:     return self.execute_command(*pieces, **options)
2012-08-24T01:34:25+00:00 app[celeryd.1]:   File &quot;/app/lib/python2.7/site-packages/kombu/transport/redis.py&quot;, line 546, in execute_command
2012-08-24T01:34:25+00:00 app[celeryd.1]:     return self.parse_response(conn, command_name, **options)
2012-08-24T01:34:25+00:00 app[celeryd.1]:   File &quot;/app/lib/python2.7/site-packages/redis/client.py&quot;, line 347, in parse_response
2012-08-24T01:34:25+00:00 app[celeryd.1]:     return self.response_callbacks[command_name](response, **options)
2012-08-24T01:34:25+00:00 app[celeryd.1]:   File &quot;/app/lib/python2.7/site-packages/redis/client.py&quot;, line 127, in zset_score_pairs
2012-08-24T01:34:25+00:00 app[celeryd.1]:     return list(izip(it, imap(score_cast_func, it)))
2012-08-24T01:34:25+00:00 app[celeryd.1]: ValueError: could not convert string to float: {&quot;body&quot;: &quot;gAJ9cQEoVQdyZXRyaWVzcQJLAFUEdGFza3EDVSZmcH
MudGFza3MuY2hlY2tfcGF5bWVudF9zdGF0dXNfY2xlYW51cHEEVQhlcnJiYWNrc3EFTlUJY2FsbGJhY2tzcQZOVQZrd2FyZ3NxB31xCFUDZXRhcQlOVQRhcmdzcQpdcQtVAmlkcQxVJDB
lMDVhYj
2012-08-24T01:34:25+00:00 app[celeryd.1]: [2012-08-23 21:34:25,500: INFO/Beat] Scheduler: Sending due task portal.tasks.decider
2012-08-24T01:34:26+00:00 app[celeryd.1]: [2012-08-23 21:34:26,492: INFO/MainProcess] Celerybeat: Shutting down...
2012-08-24T01:34:27+00:00 heroku[celeryd.1]: Process exited with status 0
2012-08-24T01:34:27+00:00 heroku[celeryd.1]: State changed from up to crashed"><pre class="notranslate"><code class="notranslate">2012-08-24T01:34:25+00:00 app[celeryd.1]: [2012-08-23 21:34:25,498: ERROR/MainProcess] Unrecoverable error: ValueError('could not convert str
ing to float: {"body": "gAJ9cQEoVQdyZXRyaWVzcQJLAFUEdGFza3EDVSZmcHMudGFza3MuY2hlY2tfcGF5bWVudF9zdGF0dXNfY2xlYW51cHEEVQhlcnJiYWNrc3EFTlUJY2Fsb
GJhY2tzcQZOVQZrd2FyZ3NxB31xCFUDZXRhcQlOVQRhcmdzcQpdcQtVAmlkcQxVJDBlMDVhYj',)
2012-08-24T01:34:25+00:00 app[celeryd.1]: Traceback (most recent call last):
2012-08-24T01:34:25+00:00 app[celeryd.1]:   File "/app/lib/python2.7/site-packages/celery/worker/__init__.py", line 353, in start
2012-08-24T01:34:25+00:00 app[celeryd.1]:     component.start()
2012-08-24T01:34:25+00:00 app[celeryd.1]:   File "/app/lib/python2.7/site-packages/celery/worker/consumer.py", line 369, in start
2012-08-24T01:34:25+00:00 app[celeryd.1]:     self.consume_messages()
2012-08-24T01:34:25+00:00 app[celeryd.1]:   File "/app/lib/python2.7/site-packages/celery/worker/consumer.py", line 449, in consume_messages
2012-08-24T01:34:25+00:00 app[celeryd.1]:     on_poll_empty()
2012-08-24T01:34:25+00:00 app[celeryd.1]:   File "/app/lib/python2.7/site-packages/kombu/transport/redis.py", line 637, in on_poll_empty
2012-08-24T01:34:25+00:00 app[celeryd.1]:     self.cycle.on_poll_empty()
2012-08-24T01:34:25+00:00 app[celeryd.1]:   File "/app/lib/python2.7/site-packages/kombu/transport/redis.py", line 209, in on_poll_empty
2012-08-24T01:34:25+00:00 app[celeryd.1]:     channel.qos.restore_visible()
2012-08-24T01:34:25+00:00 app[celeryd.1]:   File "/app/lib/python2.7/site-packages/kombu/transport/redis.py", line 102, in restore_visible
2012-08-24T01:34:25+00:00 app[celeryd.1]:     start=start, num=num, withscores=True)
2012-08-24T01:34:25+00:00 app[celeryd.1]:   File "/app/lib/python2.7/site-packages/newrelic-1.4.0.137/newrelic/api/object_wrapper.py", line 1
66, in __call__
2012-08-24T01:34:25+00:00 app[celeryd.1]:     self._nr_instance, args, kwargs)
2012-08-24T01:34:25+00:00 app[celeryd.1]:   File "/app/lib/python2.7/site-packages/newrelic-1.4.0.137/newrelic/api/function_trace.py", line 7
6, in literal_wrapper
2012-08-24T01:34:25+00:00 app[celeryd.1]:     return wrapped(*args, **kwargs)
2012-08-24T01:34:25+00:00 app[celeryd.1]:   File "/app/lib/python2.7/site-packages/redis/client.py", line 1076, in zrevrangebyscore
2012-08-24T01:34:25+00:00 app[celeryd.1]:     return self.execute_command(*pieces, **options)
2012-08-24T01:34:25+00:00 app[celeryd.1]:   File "/app/lib/python2.7/site-packages/kombu/transport/redis.py", line 546, in execute_command
2012-08-24T01:34:25+00:00 app[celeryd.1]:     return self.parse_response(conn, command_name, **options)
2012-08-24T01:34:25+00:00 app[celeryd.1]:   File "/app/lib/python2.7/site-packages/redis/client.py", line 347, in parse_response
2012-08-24T01:34:25+00:00 app[celeryd.1]:     return self.response_callbacks[command_name](response, **options)
2012-08-24T01:34:25+00:00 app[celeryd.1]:   File "/app/lib/python2.7/site-packages/redis/client.py", line 127, in zset_score_pairs
2012-08-24T01:34:25+00:00 app[celeryd.1]:     return list(izip(it, imap(score_cast_func, it)))
2012-08-24T01:34:25+00:00 app[celeryd.1]: ValueError: could not convert string to float: {"body": "gAJ9cQEoVQdyZXRyaWVzcQJLAFUEdGFza3EDVSZmcH
MudGFza3MuY2hlY2tfcGF5bWVudF9zdGF0dXNfY2xlYW51cHEEVQhlcnJiYWNrc3EFTlUJY2FsbGJhY2tzcQZOVQZrd2FyZ3NxB31xCFUDZXRhcQlOVQRhcmdzcQpdcQtVAmlkcQxVJDB
lMDVhYj
2012-08-24T01:34:25+00:00 app[celeryd.1]: [2012-08-23 21:34:25,500: INFO/Beat] Scheduler: Sending due task portal.tasks.decider
2012-08-24T01:34:26+00:00 app[celeryd.1]: [2012-08-23 21:34:26,492: INFO/MainProcess] Celerybeat: Shutting down...
2012-08-24T01:34:27+00:00 heroku[celeryd.1]: Process exited with status 0
2012-08-24T01:34:27+00:00 heroku[celeryd.1]: State changed from up to crashed
</code></pre></div>