title: Scala/Spark DataFrame: join rows to get a pair within itself
body: <p>I am trying to write a function that will tranform df -> df2 as follows:</p>
  
  <pre><code>// input dataframe df
  +-----+-----+
  |  T  |  S  |
  +-----+-----+
  |    A|   4|
  |    B|   8|
  |    C|   8|
  |    D|   2|
  +-----+-----+
  </code></pre>
  
  <p>I need a function that will take <code>df</code> as input and return <code>df2</code> as output.</p>
  
  <pre><code>// output dataframe df2
  +-----+-----+-----+
  | T1 | T2 | S=T1+T2 |
  +-----+-----+-----+
  |    A|    B|  12|
  |    A|    C|  12|
  |    A|    D|  6|
  |    B|    C|  16|
  |    B|    D|  10|
  |    C|    D|  10|
  +-----+-----+-----+
  </code></pre>
  
  <p><strong><em>EDIT</em></strong>
  I came up with this solution. Any improvement will be welcomed.</p>
  
  <pre><code>val sumOf = udf((left_score: Float, right_score: Float) =&gt; left_score + right_score)
  
  val left = df.select("T", "S").withColumnRenamed("T", "T1").withColumnRenamed("S", "S1")
  val right= df.select("T", "S").withColumnRenamed("T", "T2").withColumnRenamed("S", "S2")
  
  val joinDF = left.join(right, left.col("T1") !== right.col("T2"))
  val outDF = joinDF.withColumn("S", sumOf($"S1", $"S2")).select("T1", "T2", "S")
  </code></pre>