title: Spark DataFrame: does groupBy after orderBy maintain that order?
body: <p>I have a Spark 2.0 dataframe <code>example</code> with the following structure:</p>
  
  <pre><code>id, hour, count
  id1, 0, 12
  id1, 1, 55
  ..
  id1, 23, 44
  id2, 0, 12
  id2, 1, 89
  ..
  id2, 23, 34
  etc.
  </code></pre>
  
  <p>It contains 24 entries for each id (one for each hour of the day) and is ordered by id, hour using the orderBy function. </p>
  
  <p>I have created an Aggregator <code>groupConcat</code>:</p>
  
  <pre><code>  def groupConcat(separator: String, columnToConcat: Int) = new Aggregator[Row, String, String] with Serializable {
      override def zero: String = ""
  
      override def reduce(b: String, a: Row) = b + separator + a.get(columnToConcat)
  
      override def merge(b1: String, b2: String) = b1 + b2
  
      override def finish(b: String) = b.substring(1)
  
      override def bufferEncoder: Encoder[String] = Encoders.STRING
  
      override def outputEncoder: Encoder[String] = Encoders.STRING
    }.toColumn
  </code></pre>
  
  <p>It helps me concatenate columns into strings to obtain this final dataframe:  </p>
  
  <pre><code>id, hourly_count
  id1, 12:55:..:44
  id2, 12:89:..:34
  etc.
  </code></pre>
  
  <p>My question is, if I do <code>example.orderBy($"id",$"hour").groupBy("id").agg(groupConcat(":",2) as "hourly_count")</code>, does that guarantee that the hourly counts will be ordered correctly in their respective buckets? </p>
  
  <p>I read that this is not necessarily the case for RDDs (see <a href="https://stackoverflow.com/questions/29792320/spark-sort-by-key-and-then-group-by-to-get-ordered-iterable">Spark sort by key and then group by to get ordered iterable?</a>), but maybe it's different for DataFrames ? </p>
  
  <p>If not, how can I work around it ? </p>