{"action":"opened","issue":{"url":"https://api.github.com/repos/celery/celery/issues/3325","repository_url":"https://api.github.com/repos/celery/celery","labels_url":"https://api.github.com/repos/celery/celery/issues/3325/labels{/name}","comments_url":"https://api.github.com/repos/celery/celery/issues/3325/comments","events_url":"https://api.github.com/repos/celery/celery/issues/3325/events","html_url":"https://github.com/celery/celery/issues/3325","id":166363882,"number":3325,"title":"Celery retry entire chain","user":{"login":"matt-snider","id":11810057,"avatar_url":"https://avatars.githubusercontent.com/u/11810057?v=3","gravatar_id":"","url":"https://api.github.com/users/matt-snider","html_url":"https://github.com/matt-snider","followers_url":"https://api.github.com/users/matt-snider/followers","following_url":"https://api.github.com/users/matt-snider/following{/other_user}","gists_url":"https://api.github.com/users/matt-snider/gists{/gist_id}","starred_url":"https://api.github.com/users/matt-snider/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/matt-snider/subscriptions","organizations_url":"https://api.github.com/users/matt-snider/orgs","repos_url":"https://api.github.com/users/matt-snider/repos","events_url":"https://api.github.com/users/matt-snider/events{/privacy}","received_events_url":"https://api.github.com/users/matt-snider/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2016-07-19T15:39:23Z","updated_at":"2016-07-19T15:39:23Z","closed_at":null,"body":"Hi! \r\n\r\nI've got a situation that I've been trying to figure out for some time. I've searched quite a lot, asked in the celery IRC channel, and tinkered around to no avail. \r\n\r\nI've come up with the following contrived example to leave out actual details of the system. This can be envisioned as some system where connections come and go, and when a connection exists that matches the conditions (x and y) then we should run the second task.\r\n```python\r\n@app.task(bind=True, max_retries=None)\r\ndef find_valid_connection(x, y):\r\n      # attempt to find connection matching x and y\r\n      if not connection:\r\n           self.retry(countdown=30)\r\n      return connection\r\n\r\n@app.task(bind=True, max_retries=3)\r\ndef use_connection(connection):\r\n      try:\r\n          # attempt to use connection\r\n      except:\r\n          # retry, but connection is now out of date\r\n          self.retry(countdown=30)\r\n```\r\n\r\nWhich I use in a chain:\r\n```python\r\ntask = chain(find_valid_connection.s(1, 2), use_connection.s())\r\n#...\r\n```\r\n\r\nThe first task should run until a valid connection is found, and \"sleep\" for 30 seconds every time it searches and doesn't find one. Once it finds a connection, the bigger task runs which actually uses the resource. \r\n\r\nThe problem is, that when the second task fails and retries, the connection may no longer be valid. So essentially I need to retry the whole chain. This doesn't seem possible. Combining them into one task is also difficult because the first task needs to have `max_retries=None`. \r\n\r\nIs this something that doesn't fit well to Celery's model, or have I completely missed a feature that could be very useful here?"}}
{"actor":{"display_login":"matt-snider"}}