title: Spark dataframe explode pair of lists
body: <p>My dataframe has 2 columns which look like this:</p>
  
  <pre><code>col_id| col_name
  -----------
  id1   | name1
  id2   | name2
  ------------
  id3   | name3
  id4   | name4
  ....
  </code></pre>
  
  <p>so for each row, there are 2 matching arrays of the same length in columns id and name. What I want is to get each pair of id/name as a separate row like:</p>
  
  <pre><code>col_id| col_name
  -----------
  id1   | name1
  -----------
  id2   | name2
  ....
  </code></pre>
  
  <p><code>explode</code> seems like the function to use but I can't seem to get it to work. What I tried is:</p>
  
  <pre><code>rdd.explode(col("col_id"), col("col_name")) ({
  case row: Row =&gt;
    val ids: java.util.List[String] = row.getList(0)
    val names: java.util.List[String] = row.getList(1)
  
    var res: Array[(String, String)] = new Array[(String, String)](ids.size)
    for (i &lt;- 0 until ids.size) {
      res :+ (ids.get(i), names.get(i))
    }
    res
  })
  </code></pre>
  
  <p>This however returns only nulls so it might just be my poor knowledge of Scala. Can anyone point out the issue?</p>