title: How to make Spark streams execute sequentially
body: <h2>Issue</h2>
  <p>I have a job that executes two streams in total but I want the last one to start after the first stream has finished since the first stream saves events from the readstream in a DeltaTable that serve as input for the second stream. The problem is that what is being added in the first stream is not available in the second stream, in the current notebook run, because they start simultaneously.</p>
  <p>Is there a way to enforce the order while running it from the same notebook?</p>
  <p>I've tried the <code>awaitTermination</code> function but discovered this does not solve my problem. Some pseudocode:</p>
  <pre><code>def main():
      # Read eventhub
      metricbeat_df = spark \
          .readStream \
          .format(&quot;eventhubs&quot;) \
          .options(**eh_conf) \
          .load()
  
      # Save raw events
      metricbeat_df.writeStream \
          .trigger({&quot;once&quot;: True}) \
          .format(&quot;delta&quot;) \
          .partitionBy(&quot;year&quot;, &quot;month&quot;, &quot;day&quot;) \
          .outputMode(&quot;append&quot;) \
          .option(&quot;checkpointLocation&quot;, &quot;dbfs:/...&quot;) \
          .queryName(&quot;query1&quot;) \
          .table(&quot;my_db.raw_events&quot;)
  
      # Parse events
      metricbeat_df = spark.readStream \
          .format(&quot;delta&quot;) \
          .option(&quot;ignoreDeletes&quot;, True) \
          .table(&quot;my_db.raw_events&quot;)
    
      # *Do some transformations here*
  
      metricbeat_df.writeStream \
          .trigger({&quot;once&quot;: True}) \
          .format(&quot;delta&quot;) \
          .partitionBy(&quot;year&quot;, &quot;month&quot;, &quot;day&quot;) \
          .outputMode(&quot;append&quot;) \
          .option(&quot;checkpointLocation&quot;, &quot;dbfs:/...&quot;) \
          .queryName(&quot;query2&quot;) \
          .table(&quot;my_db.joined_bronze_events&quot;)
  </code></pre>
  <h2>TLDR</h2>
  <p>To summarize the issue: when I run the code above, <code>query1</code> and <code>query2</code> start at the same time which causes that <code>my_db.joined_bronze_events</code> is a bit behind <code>my_db.raw_events</code> because what is being added in query1 is not available in query2 in the current run (it will be in the next run of course).</p>
  <p>Is there a way to enforce that <code>query2</code> will not start until <code>query1</code> has finished while still running it in the same notebook?</p>