title: pyspark dataframe to dictionary: columns as keys and list of column values ad dict value
body: <p>Hi i have an requirement of converting a pyspark dataframe (or rdd) into a dictionary where column of dataframe will be keys and column_value_list as dictionary values.</p>
  
  <pre><code>name amt
  a    10
  b    20
  a    30
  b    40
  c    50
  </code></pre>
  
  <p>i want a dictionary like this:</p>
  
  <pre><code>new_dict = {'name':['a','b', 'a', 'b', 'c'], 'amt':[10,20,30,40,50]}
  </code></pre>
  
  <p>How can I do that, (avoiding collect on rdd is preferable solution). Thanks.</p>
  
  <p>I am also trying, will post my try in some time.</p>