title: What is going wrong with `unionAll` of Spark `DataFrame`?
body: <p>Using Spark 1.5.0 and given the following code, I expect unionAll to union <code>DataFrame</code>s based on their column name. In the code, I'm using some FunSuite for passing in SparkContext <code>sc</code>:</p>
  
  <pre><code>object Entities {
  
    case class A (a: Int, b: Int)
    case class B (b: Int, a: Int)
  
    val as = Seq(
      A(1,3),
      A(2,4)
    )
  
    val bs = Seq(
      B(5,3),
      B(6,4)
    )
  }
  
  class UnsortedTestSuite extends SparkFunSuite {
  
    configuredUnitTest("The truth test.") { sc =&gt;
      val sqlContext = new SQLContext(sc)
      import sqlContext.implicits._
      val aDF = sc.parallelize(Entities.as, 4).toDF
      val bDF = sc.parallelize(Entities.bs, 4).toDF
      aDF.show()
      bDF.show()
      aDF.unionAll(bDF).show
    }
  }
  </code></pre>
  
  <p>Output:</p>
  
  <pre><code>+---+---+
  |  a|  b|
  +---+---+
  |  1|  3|
  |  2|  4|
  +---+---+
  
  +---+---+
  |  b|  a|
  +---+---+
  |  5|  3|
  |  6|  4|
  +---+---+
  
  +---+---+
  |  a|  b|
  +---+---+
  |  1|  3|
  |  2|  4|
  |  5|  3|
  |  6|  4|
  +---+---+
  </code></pre>
  
  <p>Why does the result contain <strong>intermixed "b" and "a"</strong> columns, instead of aligning columns bases on column names? Sounds like a <strong>serious</strong> bug!?</p>