title: How to convert pyspark.rdd.PipelinedRDD to Data frame with out using collect() method in Pyspark?
body: <p>I have <code>pyspark.rdd.PipelinedRDD</code> <code>(Rdd1)</code>.
  when I am doing <code>Rdd1.collect()</code>,it is giving result like below.</p>
  <pre><code> [(10, {3: 3.616726727464709, 4: 2.9996439803387602, 5: 1.6767412921625855}),
   (1, {3: 2.016527311459324, 4: -1.5271512313750577, 5: 1.9665475696370045}),
   (2, {3: 6.230272144805092, 4: 4.033642544526678, 5: 3.1517805604906313}),
   (3, {3: -0.3924680103722977, 4: 2.9757316477407443, 5: -1.5689126834176417})]
  </code></pre>
  <p>Now I want to convert pyspark.rdd.PipelinedRDD  to Data frame with out using collect() method</p>
  <p>My final data frame should be like below. <code>df.show()</code> should be like:</p>
  <pre><code>+----------+-------+-------------------+
  |CId       |IID    |Score              |
  +----------+-------+-------------------+
  |10        |4      |2.9996439803387602 |
  |10        |5      |1.6767412921625855 |
  |10        |3      |3.616726727464709  |
  |1         |4      |-1.5271512313750577|
  |1         |5      |1.9665475696370045 |
  |1         |3      |2.016527311459324  |
  |2         |4      |4.033642544526678  |
  |2         |5      |3.1517805604906313 |
  |2         |3      |6.230272144805092  |
  |3         |4      |2.9757316477407443 |
  |3         |5      |-1.5689126834176417|
  |3         |3      |-0.3924680103722977|
  +----------+-------+-------------------+
  </code></pre>
  <p>I can achieve this converting to rdd next applying collect, iteration and finally Data frame.</p>
  <p>but now I want to convert <code>pyspark.rdd.PipelinedRDD</code> to Dataframe with out using any <code>collect()</code> method.</p>
  <p>please let me know how to achieve this?</p>