{"action":"created","issue":{"url":"https://api.github.com/repos/osm2vectortiles/osm2vectortiles/issues/382","repository_url":"https://api.github.com/repos/osm2vectortiles/osm2vectortiles","labels_url":"https://api.github.com/repos/osm2vectortiles/osm2vectortiles/issues/382/labels{/name}","comments_url":"https://api.github.com/repos/osm2vectortiles/osm2vectortiles/issues/382/comments","events_url":"https://api.github.com/repos/osm2vectortiles/osm2vectortiles/issues/382/events","html_url":"https://github.com/osm2vectortiles/osm2vectortiles/issues/382","id":167393474,"number":382,"title":"Unacknowledged messages","user":{"login":"stirringhalo","id":14035518,"avatar_url":"https://avatars.githubusercontent.com/u/14035518?v=3","gravatar_id":"","url":"https://api.github.com/users/stirringhalo","html_url":"https://github.com/stirringhalo","followers_url":"https://api.github.com/users/stirringhalo/followers","following_url":"https://api.github.com/users/stirringhalo/following{/other_user}","gists_url":"https://api.github.com/users/stirringhalo/gists{/gist_id}","starred_url":"https://api.github.com/users/stirringhalo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/stirringhalo/subscriptions","organizations_url":"https://api.github.com/users/stirringhalo/orgs","repos_url":"https://api.github.com/users/stirringhalo/repos","events_url":"https://api.github.com/users/stirringhalo/events{/privacy}","received_events_url":"https://api.github.com/users/stirringhalo/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/osm2vectortiles/osm2vectortiles/labels/question","name":"question","color":"cc317c"}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":5,"created_at":"2016-07-25T15:23:03Z","updated_at":"2016-07-27T13:45:00Z","closed_at":null,"body":"Question about handling unacknowledged messages, pardon any misunderstanding of RabbitMQ.\r\n\r\nIt seems that a fair number of messages are unacknowledged despite heartbeat being disabled. \r\n\r\nI am doing full planet renders at zoom level 6, so there are 4097 tiles which on average take an hour (but some take much longer).\r\n\r\nGenerally for a full planet, I have 8-12 unacknowledged messages, which has recently jumped to 60.\r\n\r\nHere are my settings: \r\n\r\n- blocked_connection_timeout=1200\r\n- render_timeout= 54000\r\n- tile_timeout = 54000000\r\n\r\nWhat's the best way to requeue those unack'd messaged? Kill all the connections in RabbitMQ and startup the cluster again? \r\n\r\nor...\r\n\r\n`try:\r\n\r\n            channel.basic_ack(delivery_tag=method_frame.delivery_tag)\r\n\r\n            result_msg = handle_message(\r\n                tm2source, bucket,\r\n                functools.partial(s3_url, host, port, bucket_name),\r\n                body\r\n            )\r\n\r\n            durable_publish(channel, result_queue_name,\r\n                            body=json.dumps(result_msg))\r\n        except:\r\n            channel.basic_ack(delivery_tag=method_frame.delivery_tag)\r\n            durable_publish(channel, failed_queue_name, body=body)\r\n            channel.stop_consuming()\r\n            time.sleep(5)  # Give RabbitMQ some time\r\n            raise\r\n`\r\n\r\nMove the ack ahead of the tiling? That way, the message gets immediately acknowledged. If it dies later in tiling, it still fails the try and the message goes to the failed queue.\r\n\r\nThoughts?\r\n"},"comment":{"url":"https://api.github.com/repos/osm2vectortiles/osm2vectortiles/issues/comments/235589210","html_url":"https://github.com/osm2vectortiles/osm2vectortiles/issues/382#issuecomment-235589210","issue_url":"https://api.github.com/repos/osm2vectortiles/osm2vectortiles/issues/382","id":235589210,"user":{"login":"stirringhalo","id":14035518,"avatar_url":"https://avatars.githubusercontent.com/u/14035518?v=3","gravatar_id":"","url":"https://api.github.com/users/stirringhalo","html_url":"https://github.com/stirringhalo","followers_url":"https://api.github.com/users/stirringhalo/followers","following_url":"https://api.github.com/users/stirringhalo/following{/other_user}","gists_url":"https://api.github.com/users/stirringhalo/gists{/gist_id}","starred_url":"https://api.github.com/users/stirringhalo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/stirringhalo/subscriptions","organizations_url":"https://api.github.com/users/stirringhalo/orgs","repos_url":"https://api.github.com/users/stirringhalo/repos","events_url":"https://api.github.com/users/stirringhalo/events{/privacy}","received_events_url":"https://api.github.com/users/stirringhalo/received_events","type":"User","site_admin":false},"created_at":"2016-07-27T13:45:00Z","updated_at":"2016-07-27T13:45:00Z","body":"> How to proceed\r\n>\r\n> 2 alternatives.\r\n>\r\n> Fix RabbitMQ process\r\n>\r\n> Pros:\r\n>\r\n> Components still work\r\n> I really like working with the pipe tools to administer the jobs\r\n> Cons:\r\n> \r\n> In the end it will be a hacky solution\r\n> A lot of time has already been spent solving this problem - perhaps dead end\r\n> Different job queue -> beanstalkd\r\n>\r\n> Pros:\r\n>\r\n> Solving the timeout thing once and for all\r\n> Cons:\r\n>\r\n> Rewrite merge-jobs, generate-jobs and export-worker\r\n> Option 2 is more feasible now than earlier because now I have no deadline of finishing up the thesis. And can invest time in stuff like this.\r\n> I know @klokan is for option 2 as well.\r\n\r\nWithout knowing much about beanstalkd, I'm personally in favour of RabbitMQ as its a known quantity with known issues. We also have less breaking commits. "}}
{"actor":{"display_login":"stirringhalo"}}