title: Split Spark Dataframe string column into multiple columns
body: <p>I've seen various people suggesting that <code>Dataframe.explode</code> is a useful way to do this, but it results in more rows than the original dataframe, which isn't what I want at all. I simply want to do the Dataframe equivalent of the very simple:</p>
  
  <pre><code>rdd.map(lambda row: row + [row.my_str_col.split('-')])
  </code></pre>
  
  <p>which takes something looking like:</p>
  
  <pre><code>col1 | my_str_col
  -----+-----------
    18 |  856-yygrm
   201 |  777-psgdg
  </code></pre>
  
  <p>and converts it to this:</p>
  
  <pre><code>col1 | my_str_col | _col3 | _col4
  -----+------------+-------+------
    18 |  856-yygrm |   856 | yygrm
   201 |  777-psgdg |   777 | psgdg
  </code></pre>
  
  <p>I am aware of <code>pyspark.sql.functions.split()</code>, but it results in a nested array column instead of two top-level columns like I want. </p>
  
  <p>Ideally, I want these new columns to be named as well.</p>