title: Why imap and write is faster than apply_async and write?
body: <p>imap version:</p>
  
  <pre><code>import os
  import multiprocessing as mp
  import timeit
  import string
  import random
  
  
  PROCESSES = 5
  FILE = 'test_imap.txt'
  
  
  
  def remove_file():
      try:
          os.remove(FILE)
      except FileNotFoundError:
          pass
  
  
  def produce(i):
      return [''.join(random.choice(string.ascii_uppercase + string.digits) for _ in range(32)) for i in range(100000)]
  
  
  def imap_version():
      with mp.Pool(PROCESSES) as p:
          with open(FILE, 'a') as fp:
              for lines in p.imap_unordered(produce, range(5)):
                  for line in lines:
                      fp.write(line + '\n')
  
  
  if __name__ == '__main__':
      remove_file()
      imap_version_result = timeit.repeat("imap_version()", setup="from __main__ import imap_version", repeat=5, number=5)
      print('imap result:', imap_version_result)
  </code></pre>
  
  <p>apply_async version:</p>
  
  <pre><code>import os
  import multiprocessing as mp
  import timeit
  import string
  import random
  
  
  PROCESSES = 5
  FILE = 'test_apply.txt'
  
  
  
  def remove_file():
      try:
          os.remove(FILE)
      except FileNotFoundError:
          pass
  
  
  def produce():
      return [''.join(random.choice(string.ascii_uppercase + string.digits) for _ in range(32)) for i in range(100000)]
  
  
  def worker():
      lines = produce()
      with open(FILE, 'a') as fp:
          for line in lines:
              fp.write(line + '\n')
  
  
  def apply_version():
      with mp.Pool(PROCESSES) as p:
          processes = []
          for i in range(5):
              processes.append(p.apply_async(worker))
  
          while True:
              if all((p.ready() for p in processes)):
                  break
  
  
  if __name__ == '__main__':
      remove_file()
      apply_version_result = timeit.repeat("apply_version()", setup="from __main__ import apply_version", repeat=5, number=5)
      print('apply result', apply_version_result)
  </code></pre>
  
  <p>Results:</p>
  
  <pre><code>imap result: [62.71130559899029, 62.65627204600605, 62.534730065002805, 62.67373917000077, 62.74415319500258]
  apply result [72.03727042900573, 72.17959955699916, 72.2304800950078, 72.02653418600676, 72.11620796499483]
  </code></pre>
  
  <p>I expected <code>imap</code> to be slower because child processes need to pickle the results to the main process and then write to file, whereas each child process in <code>apply_async</code> directly write the results to file. Instead, <code>imap</code> is slower than <code>apply_async</code>.</p>
  
  <p>Why is this so?</p>
  
  <p>nb: This was done using Python 3.4.3 on Mac OS X 10.11</p>