title: How to use mapPartitions in Spark Scala?
body: <p>I have DocsRDD : RDD[String, String]</p>
  
  <pre><code>val DocsRDD = sc.wholeTextFiles("myDirectory/*" , 2)
  </code></pre>
  
  <p>DocsRDD:</p>
  
  <pre><code>Doc1.txt , bla bla bla .....\n bla bla bla \n bla ... bla
  Doc2.txt , bla bla bla .....bla \n bla bla \n bla ... bla
  Doc3.txt , bla bla bla .....\n bla bla bla \n bla ... bla
  Doc4.txt , bla bla \n  .....\n bla bla bla bla \n ... bla
  </code></pre>
  
  <p>Is there an efficient, elegant way to extract n-grams from these with mapPartitions?
  So far i have tried everything, i have read everything i could find at least 5 times over and over about mapPartitions but i still cannot understand how to use it! It seems waaay too difficult to manipulate.
  In short i want :</p>
  
  <pre><code>val NGramsRDD = DocsRDD.map(x =&gt; (x._1 , x._2.sliding(n) ) )
  </code></pre>
  
  <p>but efficiently with mapPartitions.
  My basic misunderstanding of mapPartitions is :</p>
  
  <p>OneDocRDD : RDD[String]</p>
  
  <pre><code> val OneDocRDD = sc.textFile("myDoc1.txt" , 2)
                     .mapPartitions(s1 : Iterator[String] =&gt; s2 : Iterator[String])
  </code></pre>
  
  <p>I Cannot understand this! From when s1 was Iterator[String]? s1 is String after sc.textfile.</p>
  
  <p>Alright my second question is : Will mapPartitions improve my overcome against map in this situation?</p>
  
  <p>Last but not Least important: 
  can f() be :</p>
  
  <pre><code>     f(Iterator[String]) : Iterator[Something else?]
  </code></pre>