title: Spark SQL: Window function lag until a condition met
body: <p>I am processing this dataset in spark:</p>
  
  <pre><code>+------------+------------+------------+
  |     ColumnA|     ColumnB|     Result |
  +------------+------------+------------+
  |      ABCDEF|    MNOPQRST|      true  |
  |      123455|      UVWXYZ|      false |
  |      ABCDEF|    MNOPQRST|      false | (should be true)
  |      123455|      UVWXYZ|      false |
  |      123455|      UVWXYZ|      false |
  |      ABCDEF|    EFGHIJK |      false |
  +------------+------------+------------+
  </code></pre>
  
  <p>Rules are:</p>
  
  <ol>
  <li>If rank is 1 for the give partition set <code>Result</code> as true.</li>
  <li>if the rank is not 1 and <code>ColumnA</code> value is <code>123455</code> set <code>Result</code> value as <code>false</code></li>
  <li><p>if the rank is not 1 and <code>ColumnA</code> value is not <code>123455</code> and if <code>ColumnB</code> value matches with previous row's <code>ColumnB</code> value, set Result as true. Make sure that the previous row's ColumnA's value is not <code>123455</code></p>
  
  <p>WindowSpec w = Window.partitionBy("ColumnA, ColumnB");</p>
  
  <p>Column matchColumnB = functions.col("ColumnB").equalTo(
       functions.lag("ColumnB", 1).over(w));</p></li>
  </ol>
  
  <p>Here the window function checks the previous row without considering the previous row's ColumnA value.</p>
  
  <p>For example in the above dataset, Row 3's ColumnB value should be compared with Row1 not the Row2.</p>
  
  <p>I tried looking at <code>Window.unboundedPreceding</code> but not sure how to use it in this scenario.</p>
  
  <p>Is there a way to achieve this?</p>