title: Spark - Aggregate user activity to represent periods
body: <p>I'm working with a Spark DataFrame like the one below:</p>
  
  <pre><code>user | 1 | 2 | 3 | 4 | ... | 53
  -------------------------------
     1 | 1 | 0 | 0 | 1 | ... |  1
     2 | 0 | 1 | 1 | 1 | ... |  0
     3 | 1 | 1 | 0 | 0 | ... |  1
     .
     .
     .
     n | 1 | 0 | 1 | 1 | ... |  0
  </code></pre>
  
  <p>which has columns representing user ID, then a column for each week of the year containing a boolean, representing whether a user was active for that week. </p>
  
  <p>My goal is to reduce this to a table like so:</p>
  
  <pre><code>user | active_start | active_end | duration
  -------------------------------------------
     1 |            1 |          1 |        1
     1 |            4 |          4 |        1
     1 |           53 |         53 |        1
     2 |            2 |          4 |        3
     3 |            1 |          2 |        2
     3 |           53 |         53 |        1
     .
     .
     .
     n |            1 |          1 |        1
     n |            3 |          4 |        2
  </code></pre>
  
  <p>which contains periods of continuous activity.</p>
  
  <p>I'm at somewhat of a loss as to how I should manipulate the table/aggregate the values so as to create a new row when a gap is detected. </p>
  
  <p>I have tried using code for Island/Gap detection to generate these groups, but have been unable to implement a version which does not detect and generate rows for smaller sub-islands within larger ones.</p>
  
  <p>Any help would be appreciated,
  Thanks!</p>