title: Exploding nested Struct in Spark dataframe
body: <p>I'm working through a Databricks example. The schema for the dataframe looks like:</p>
  <pre><code>&gt; parquetDF.printSchema
  root
  |-- department: struct (nullable = true)
  |    |-- id: string (nullable = true)
  |    |-- name: string (nullable = true)
  |-- employees: array (nullable = true)
  |    |-- element: struct (containsNull = true)
  |    |    |-- firstName: string (nullable = true)
  |    |    |-- lastName: string (nullable = true)
  |    |    |-- email: string (nullable = true)
  |    |    |-- salary: integer (nullable = true)
  </code></pre>
  <p>In the example, they show how to explode the employees column into 4 additional columns:</p>
  <pre><code>val explodeDF = parquetDF.explode($&quot;employees&quot;) { 
  case Row(employee: Seq[Row]) =&gt; employee.map{ employee =&gt;
    val firstName = employee(0).asInstanceOf[String]
    val lastName = employee(1).asInstanceOf[String]
    val email = employee(2).asInstanceOf[String]
    val salary = employee(3).asInstanceOf[Int]
    Employee(firstName, lastName, email, salary)
   }
  }.cache()
  display(explodeDF)
  </code></pre>
  <p>How would I do something similar with the department column (i.e. add two additional columns to the dataframe called &quot;id&quot; and &quot;name&quot;)? The methods aren't exactly the same, and I can only figure out how to create a brand new data frame using:</p>
  <pre><code>val explodeDF = parquetDF.select(&quot;department.id&quot;,&quot;department.name&quot;)
  display(explodeDF)
  </code></pre>
  <p>If I try:</p>
  <pre><code>val explodeDF = parquetDF.explode($&quot;department&quot;) { 
    case Row(dept: Seq[String]) =&gt; dept.map{dept =&gt; 
    val id = dept(0) 
    val name = dept(1)
    } 
  }.cache()
  display(explodeDF)
  </code></pre>
  <p>I get the warning and error:</p>
  <pre><code>&lt;console&gt;:38: warning: non-variable type argument String in type pattern Seq[String] is unchecked since it is eliminated by erasure
              case Row(dept: Seq[String]) =&gt; dept.map{dept =&gt; 
                             ^
  &lt;console&gt;:37: error: inferred type arguments [Unit] do not conform to    method explode's type parameter bounds [A &lt;: Product]
    val explodeDF = parquetDF.explode($&quot;department&quot;) { 
                                     ^
  </code></pre>