title: List (or iterator) of tuples returned by MAP (PySpark)
body: <p>I have a mapper method:</p>
  
  <pre><code>def mapper(value):
      ...
      for key, value in some_list:
          yield key, value
  </code></pre>
  
  <p>what I need is not really far from the ordinary wordcount example, actually. I already have working script, but only if the mapper method looks like that:</p>
  
  <pre><code>def mapper(value):
      ...
      return key, value
  </code></pre>
  
  <p>This is how its call looks like:</p>
  
  <pre><code>sc.textFile(sys.argv[2], 1).map(mapper).reduceByKey(reducer).collect()
  </code></pre>
  
  <p>I spent 2 hours trying to write code that would support generators in mapper. But couldn't do that. I even agree to just returning a list:</p>
  
  <pre><code>def mapper(value):
      ...
      result_list = []
      for key, value in some_list:
          result_list.append( key, value )
      return result_list
  </code></pre>
  
  <p>Here: <a href="https://groups.google.com/forum/#!searchin/spark-users/flatmap" rel="noreferrer">https://groups.google.com/forum/#!searchin/spark-users/flatmap</a>$20multiple/spark-users/1WqVhRBaJsU/-D5QRbenlUgJ
  I found that I should use flatMap, but it didn't do the trick - my reducer then started to get inputs like (key1, value1, key2, value2, value3, ...) - but it should be [(key1, value1), (key2, value2, value3)...]. In other words, reducer started taking only single pieces, and don't know whether it's a value or a key, and if value - to which key it belongs.</p>
  
  <p>So how to use mappers that return iterators or lists?</p>
  
  <p>Thanks!</p>