title: Scheduling celery tasks with large ETA
body: <p>I am currently experimenting with future tasks in celery using the <a href="http://docs.celeryproject.org/en/latest/userguide/calling.html#eta-and-countdown" rel="nofollow noreferrer" title="ETA">ETA</a> feature and a redis broker. One of the known issues with using a redis broker has to do with the <a href="http://docs.celeryproject.org/en/latest/getting-started/brokers/redis.html#id1" rel="nofollow noreferrer">visibility timeout</a>:</p>
  <blockquote>
  <p>If a task isn’t acknowledged within the Visibility Timeout the task will be redelivered to another worker and executed.</p>
  <p>This causes problems with ETA/countdown/retry tasks where the time to execute exceeds the visibility timeout; in fact if that happens it will be executed again, and again in a loop.</p>
  </blockquote>
  <p>Some tasks that I can envision will have an ETA on the timescale of weeks/months. Setting the <code>visibility timeout</code> large enough to encompass these tasks is probably unwise.</p>
  <p>Are there any paths forward for processing these tasks with a redis broker? I am aware of <a href="https://stackoverflow.com/questions/41300209/celery-sqs-duplication-of-tasks-sqs-visibility-timeout?rq=1">this question</a>. Is changing brokers the only option?</p>