title: How to update a Static Dataframe with Streaming Dataframe in Spark structured streaming
body: <p>I have a Static <code>DataFrame</code> with millions of rows as follows.</p>
  
  <p>Static <code>DataFrame</code> : </p>
  
  <pre><code>--------------
  id|time_stamp|
  --------------
  |1|1540527851|
  |2|1540525602|
  |3|1530529187|
  |4|1520529185|
  |5|1510529182|
  |6|1578945709|
  --------------
  </code></pre>
  
  <p>Now in every batch, a Streaming <code>DataFrame</code> is being formed which contains id and updated time_stamp after some operations like below.</p>
  
  <p>In first Batch :   </p>
  
  <pre><code>--------------
  id|time_stamp|
  --------------
  |1|1540527888|
  |2|1540525999|
  |3|1530529784|
  --------------
  </code></pre>
  
  <p>Now in every batch, I want to update the Static DataFrame with the updated values of Streaming Dataframe like follows. <strong>How to do that?</strong></p>
  
  <p>Static DF after first batch : </p>
  
  <pre><code>--------------
  id|time_stamp|
  --------------
  |1|1540527888|
  |2|1540525999|
  |3|1530529784|
  |4|1520529185|
  |5|1510529182|
  |6|1578945709|
  --------------
  </code></pre>
  
  <p>I've already tried <strong>except(), union() or 'left_anti' join</strong>. But it seems <strong>structured streaming doesn't support such operations</strong>.</p>