title: How to manually run Airflow DAG on a particular directory
body: <p>I am evaluating whether Airflow is suitable for my needs (in bioinformatics).  I am having some difficulty with the Airflow model. Specifically:</p>
  
  <ul>
  <li>Where does the DAG file actually get executed?  What is its context?  How can I pass input data into the DAG definition file?  (E.g., I want ot create a task for each file in a directory.)</li>
  <li>How do I execute a DAG on an ad hoc basis?  How do I pass parameters for the DAG construction?</li>
  </ul>
  
  <p>Here is an example of what I would like to execute.  Say I just received some data as a directory containing 20 files available in some shared filesystem.  I want to execute a DAG pipeline which runs a particular bash command on each of the 20 files, then combines some of the results and performs further processing.  The DAG needs the path on the filesystem and also to list the files in the directory to construct a task for each one.</p>
  
  <p>It's probably not necessary for me to pass metadata from one task to another (which I understand is possible through <code>XCom</code>), as long as I can dynamically construct the entire DAG upfront.  But it's not clear to me how I can pass a path to the DAG construction.</p>
  
  <p>Put another way, I'd like my DAG definition to include something like</p>
  
  <pre><code>dag = DAG(...)
  for file in glob(input_path):
      t = BashOperator(..., dag=dag)
  </code></pre>
  
  <p>How do I get <code>input_path</code> passed in when I want to manually trigger a DAG?</p>
  
  <p>I also don't really have need for the cron-style scheduling.</p>