title: create empty array-column of given schema in Spark
body: <p>Due to the fact that parquet cannt parsists empty arrays, I replaced empty arrays with null before writing a table. Now as I read the table, I want to do the opposite:</p>
  
  <p>I have a DataFrame with the following schema : </p>
  
  <pre><code>|-- id: long (nullable = false)
   |-- arr: array (nullable = true)
   |    |-- element: struct (containsNull = true)
   |    |    |-- x: double (nullable = true)
   |    |    |-- y: double (nullable = true)
  </code></pre>
  
  <p>and the following content:</p>
  
  <pre><code>+---+-----------+
  | id|        arr|
  +---+-----------+
  |  1|[[1.0,2.0]]|
  |  2|       null|
  +---+-----------+
  </code></pre>
  
  <p>I'd like to replace the null-array (id=2) with an empty array, i.e.</p>
  
  <pre><code>+---+-----------+
  | id|        arr|
  +---+-----------+
  |  1|[[1.0,2.0]]|
  |  2|         []|
  +---+-----------+
  </code></pre>
  
  <p>I've tried:</p>
  
  <pre><code>val arrSchema = df.schema(1).dataType
  
  df
  .withColumn("arr",when($"arr".isNull,array().cast(arrSchema)).otherwise($"arr"))
  .show()
  </code></pre>
  
  <p>which gives : </p>
  
  <blockquote>
    <p>java.lang.ClassCastException: org.apache.spark.sql.types.NullType$
    cannot be cast to org.apache.spark.sql.types.StructType</p>
  </blockquote>
  
  <p>Edit : I don't want to "hardcode" any schema of my array column (at least not the schema of the struct) because this can vary from case to case. I can only use the schema information from <code>df</code> at runtime</p>
  
  <p>I'm using Spark 2.1 by the way, therefore I cannot use <code>typedLit</code></p>