title: Airflow Sensor - timeout
body: <p>tl;dr, Problem framing:</p>
  <p>Assuming I have a sensor poking with <code>timeout = 24*60*60</code>. Since the connection does time out occasionally, <code>retries</code> must be allowed. If the sensor now retries, the <code>timeout</code> variable is being applied to every new try with the initial <code>24*60*60</code>, and, therefore, the task does not time out after 24 hrs as it was intended.</p>
  <p>Question:</p>
  <p>Is there a way to restrict the max-time of a task - like a meta-timeout?</p>
  <p>Airflow-Version: <code>1.10.14</code></p>
  <p>Walk-thorough-thru:</p>
  <pre><code>BASE_DIR = &quot;/some/base/dir/&quot;
  FILE_NAME = &quot;some_file.xlsx&quot;
  VOL_BASE_DIR = &quot;/some/mounted/vol/&quot;
  
  default_args = {
      &quot;owner&quot;: &quot;airflow&quot;,
      &quot;depends_on_past&quot;: False,
      &quot;start_date&quot;: &quot;2020-11-01&quot;,
      &quot;retries&quot;: 2,
      &quot;retry_delay&quot;: timedelta(minutes=5),
  }
  
  dag = DAG(
      &quot;supplier&quot;,
      default_args=default_args,
      description=&quot;ETL Process for Supplier&quot;,
      schedule_interval=None,
      catchup=False,
      max_active_runs=1,
  )
  
  file_sensor =  FileSensor(
      task_id=&quot;file_sensor&quot;,
      poke_interval=60*60,
      timeout=24*60*60,
      retries=4,
      mode=&quot;reschedule&quot;,
      filepath=os.path.join(BASE_DIR,FILE_NAME)
      fs_conn_id='conn_filesensor',
      dag=dag,
  )
  
  clean_docker_vol = InitCleanProcFolderOperator(
      task_id=&quot;clean_docker_vol&quot;,
      folder=VOL_BASE_DIR,
      dag=dag,
  )
  
  ....
  
  </code></pre>
  <p>This DAG should run and check if a file exists. If it exists, it should continue. Occasionally, it can happen that the sensor-task is being rescheduled due to the file being provided too late (or, say, connection errors). The MAX-overall 'run-time' of the dag should NOT exceed 24 hrs. Due to the retries, however, the time does exceed the 24 hrs timeout, if the tasks fails and is being rescheduled.</p>
  <p>Example:</p>
  <ol>
  <li>runs for 4 hrs (18 hrs should be left)</li>
  <li>fails</li>
  <li>up_for_retry</li>
  <li>starts again with 24 hrs timeout, not 18 hrs.</li>
  </ol>
  <p>As I need to allow retries, there is not the option of just setting retries to 0 to avoid this behavior. I was rather looking for a meta-timeout variable of airflow, a hint how this can be implemented within the related classes or any other workarounds.</p>
  <p>many thanks.</p>