title: apache airflow and idempotency of file moving
body: <p>i have a workflow that i need to manage with airflow (or any other workflow engine i guess). the main issue is that the workflow needs to be started when a few files are dumped onto the file system (technically, it will be a samba mount). the first step would be to move the files onto something more permanent.</p>
  
  <p>now, as there are no hooks that will tell me when new files are being dumped in, i'm left with the problem of how to create the workflow such that if the data move takes too long, the next run of the workflow doesn't attempt to move the data again (potentially clobbering the original file move).</p>
  
  <p>my initial thoughts were to rename the files locally on the samba mount and only scan for the other files. however, should the transfer fail in any way, i would have to rename the files back so that the next run could try again.</p>
  
  <p>is there a better way to make this idempotent? is there a notion of a try/except/finally block in airflow?</p>