title: How to coalesce array columns in Spark dataframe
body: <p>Suppose I have the following dataframe:</p>
  
  <pre><code>id | myStruct
  ___________________
  1  | [val1, val2]
  ___________________
  2  | [val3, val4]
  ___________________
  1  | [val5, val6]
  </code></pre>
  
  <p>I would like to group all myStructs that share the same id into an array column of myStructs. So, the above dataframe should become</p>
  
  <pre><code>id | myStruct
  __________________________________
  1  | [[val1, val2], [val5, val6]]
  __________________________________
  2  | [[val3, val4]]
  </code></pre>
  
  <p>I know there is an array function, but that only converts each column into an array of size 1. How do I coalesce the resulting arrays?</p>
  
  <p>I am using Spark 1.5.2 in a Scala shell.</p>
  
  <p>Given that I am using Spark 1.5.2, I cannot use collect_list or collect_set.</p>