title: How to use mapPartitions in pyspark
body: <p>After following the Apache Spark documentation, I tried to experiment with the mapPartition module. In the following code, I expected to see initial RDD as in the function <code>myfunc</code> I am just returning back the iterator after printing the values. But when I do <code>collect</code> on the RDD it is empty. </p>
  
  <pre><code>from pyspark import SparkConf
  from pyspark import SparkContext                                                          
  
  def myfunc(it):
      print(it.next())
      return it
  
  def fun1(sc):
      n = 5
      rdd = sc.parallelize([x for x in range(n+1)], n)
      print(rdd.mapPartitions(myfunc).collect())
  
  
  if __name__ == "__main__":                                                                
      conf = SparkConf().setMaster("local[*]")                                              
      conf = conf.setAppName("TEST2")                                                       
      sc = SparkContext(conf = conf)                                                        
      fun1(sc)
  </code></pre>