title: Dataframe - groupby access previous to previous record in the groupedDataSet
body: <p>I have a scenario, where I need to compute <code>PreviousPolicyNo</code> based on <code>PolicyNo</code>. The data first needs to be sorted on the basis of <code>TransactionDate</code>. Then, we will have to check that if such a <code>PolicyTerm</code> was available before or not, if so, take latest available <code>PolicyNo</code> (available prior to <code>TransactionDate</code>) from that record and make entry to <code>PreviousPolicyNo</code>. If such a <code>PolicyTerm</code> was not available in the past, then pick the latest <code>PolicyNo</code> available (so far) or mark the entry as null if no previous term exist earlier (first row).</p>
  
  <p>For example,</p>
  
  <p><a href="https://i.stack.imgur.com/RGDNM.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/RGDNM.png" alt="enter image description here"></a></p>
  
  <p>To calculate, second last entry, I had to look for last value available so far (prior to 12-12-2014) for <code>PolicyTerm</code> - 2 and similarly, for last entry, I had to look for last record available (prior to 12-12-2014) with <code>PolicyTerm</code> 3.</p>
  
  <p>I need help for Scala implementation, I implemented an <code>UDF</code> (where I collected these columns in a <code>List[Struct]</code>), but it is not working good with huge dataset.</p>