title: Compare Value of Current and Previous Row in Spark
body: <p>I am trying to compare record of current and previous row in the below <code>DataFrame</code>. I want to calculate the Amount column.</p>
  
  <pre><code>scala&gt; val dataset = sc.parallelize(Seq((1, 123, 50), (2, 456, 30), (3, 456, 70), (4, 789, 80))).toDF("SL_NO","ID","AMOUNT")
  
  scala&gt; dataset.show
  +-----+---+------+
  |SL_NO| ID|AMOUNT|
  +-----+---+------+
  |    1|123|    50|
  |    2|456|    30|
  |    3|456|    70|
  |    4|789|    80|
  +-----+---+------+
  </code></pre>
  
  <p><strong>Calculation Logic:</strong></p>
  
  <ol>
  <li>For the row no 1, AMOUNT should be 50 from first row.</li>
  <li>For the row no 2, if ID of SL_NO - 2 and 1 is not same then need to consider 
  AMOUNT of SL_NO - 2 (i.e - 30). Otherwise AMOUNT of SL_NO - 1 (i.e. - 50)</li>
  <li>For the row no 3, if ID of SL_NO - 3 and 2 is not same then need to consider 
  AMOUNT of SL_NO - 3 (i.e - 70). Otherwise AMOUNT of SL_NO - 2 (i.e. - 30)</li>
  </ol>
  
  <p>Same logic need to follow for the other rows also.</p>
  
  <p><strong>Expected Output:</strong></p>
  
  <pre><code>+-----+---+------+
  |SL_NO| ID|AMOUNT|
  +-----+---+------+
  |    1|123|    50|
  |    2|456|    30|
  |    3|456|    30|
  |    4|789|    80|
  +-----+---+------+
  </code></pre>
  
  <p>Please help.</p>