title: Running airflow tasks/dags in parallel
body: <p>I'm using airflow to orchestrate some python scripts. I have a "main" dag from which several subdags are run. My main dag is supposed to run according to the following overview:</p>
  
  <p><a href="https://i.stack.imgur.com/X8sNT.png" rel="noreferrer"><img src="https://i.stack.imgur.com/X8sNT.png" alt="enter image description here"></a></p>
  
  <p>I've managed to get to this structure in my main dag by using the following lines:</p>
  
  <pre><code>etl_internal_sub_dag1 &gt;&gt; etl_internal_sub_dag2 &gt;&gt; etl_internal_sub_dag3
  etl_internal_sub_dag3 &gt;&gt; etl_adzuna_sub_dag
  etl_internal_sub_dag3 &gt;&gt; etl_adwords_sub_dag
  etl_internal_sub_dag3 &gt;&gt; etl_facebook_sub_dag
  etl_internal_sub_dag3 &gt;&gt; etl_pagespeed_sub_dag
  
  etl_adzuna_sub_dag &gt;&gt; etl_combine_sub_dag
  etl_adwords_sub_dag &gt;&gt; etl_combine_sub_dag
  etl_facebook_sub_dag &gt;&gt; etl_combine_sub_dag
  etl_pagespeed_sub_dag &gt;&gt; etl_combine_sub_dag
  </code></pre>
  
  <p>What I want airflow to do is to first run the <code>etl_internal_sub_dag1</code> then the <code>etl_internal_sub_dag2</code> and then the <code>etl_internal_sub_dag3</code>. When the <code>etl_internal_sub_dag3</code> is finished I want <code>etl_adzuna_sub_dag</code>, <code>etl_adwords_sub_dag</code>, <code>etl_facebook_sub_dag</code>, and <code>etl_pagespeed_sub_dag</code> to run in parallel. Finally, when these last four scripts are finished, I want the <code>etl_combine_sub_dag</code> to run. </p>
  
  <p>However, when I run the main dag, <code>etl_adzuna_sub_dag</code>, <code>etl_adwords_sub_dag</code>, <code>etl_facebook_sub_dag</code>, and <code>etl_pagespeed_sub_dag</code>  are run one by one and not in parallel.</p>
  
  <p><strong>Question:</strong> How do I make sure that the scripts <code>etl_adzuna_sub_dag</code>, <code>etl_adwords_sub_dag</code>, <code>etl_facebook_sub_dag</code>, and <code>etl_pagespeed_sub_dag</code> are run in parallel?</p>
  
  <p><strong>Edit:</strong> My <code>default_args</code> and <code>DAG</code> look like this:</p>
  
  <pre><code>default_args = {
      'owner': 'airflow',
      'depends_on_past': False,
      'start_date': start_date,
      'end_date': end_date,
      'email': ['myname@gmail.com'],
      'email_on_failure': False,
      'email_on_retry': False,
      'retries': 0,
      'retry_delay': timedelta(minutes=5),
  }
  
  DAG_NAME = 'main_dag'
  
  dag = DAG(DAG_NAME, default_args=default_args, catchup = False)
  </code></pre>