title: Cumulate arrays from earlier rows (PySpark dataframe)
body: <p>A (Python) example will make my question clear. Let's say I have a Spark dataframe of people who watched certain movies on certain dates, as follows:</p>
  
  <pre class="lang-py prettyprint-override"><code>movierecord = spark.createDataFrame([("Alice", 1, ["Avatar"]),("Bob", 2, ["Fargo", "Tron"]),("Alice", 4, ["Babe"]), ("Alice", 6, ["Avatar", "Airplane"]), ("Alice", 7, ["Pulp Fiction"]), ("Bob", 9, ["Star Wars"])],["name","unixdate","movies"])
  </code></pre>
  
  <p>The schema and the dataframe defined by the above look as follows:</p>
  
  <pre><code>root
   |-- name: string (nullable = true)
   |-- unixdate: long (nullable = true)
   |-- movies: array (nullable = true)
   |    |-- element: string (containsNull = true)
  
  +-----+--------+------------------+
  |name |unixdate|movies            |
  +-----+--------+------------------+
  |Alice|1       |[Avatar]          |
  |Bob  |2       |[Fargo, Tron]     |
  |Alice|4       |[Babe]            |
  |Alice|6       |[Avatar, Airplane]|
  |Alice|7       |[Pulp Fiction]    |
  |Bob  |9       |[Star Wars]       |
  +-----+--------+------------------+
  </code></pre>
  
  <p>I'd like to go from the above to generating a new dataframe column which holds <strong>all <em>previous</em> movies seen by each user</strong>, without duplicates ("previous" per the unixdate field). So it should look like this:</p>
  
  <pre><code>+-----+--------+------------------+------------------------+
  |name |unixdate|movies            |previous_movies         |
  +-----+--------+------------------+------------------------+
  |Alice|1       |[Avatar]          |[]                      |
  |Bob  |2       |[Fargo, Tron]     |[]                      |
  |Alice|4       |[Babe]            |[Avatar]                |
  |Alice|6       |[Avatar, Airplane]|[Avatar, Babe]          |
  |Alice|7       |[Pulp Fiction]    |[Avatar, Babe, Airplane]|
  |Bob  |9       |[Star Wars]       |[Fargo, Tron]           |
  +-----+--------+------------------+------------------------+
  </code></pre>
  
  <p>How do I implement this in a nice efficient way?</p>