title: Windows Python 2.7 multiprocessing.Manager.Queue deadlocks on put from child process
body: <p>I'm trying to get code similar to the following example working correctly:</p>
  
  <pre><code>from multiprocessing import Process, Queue, Manager, Pool
  import time
  from datetime import datetime
  
  def results_producer(the_work, num_procs):
      results = Manager().Queue()
      ppool = Pool(num_procs)
      multiplier = 3
      #step = len(the_work)/(num_procs*multiplier)
      step = 100
      for i in xrange(0,len(the_work), step):
          batch = the_work[i:i+step]
          ppool.apply_async(do_work1, args=(i,batch,results))#,callback=results.put_nowait)
      return (ppool, results)
  
  def results_consumer(results, total_work, num_procs, pool=None):
      current = 0
      batch_size=10
      total = total_work
      est_remaining = 0
      while current &lt; total_work:
          size = results.qsize()
          est_remaining = total_work - (current + size)
          if current % 1000 == 0:
              print 'Attempting to retrieve item from queue that is empty? %s, with size: %d and remaining work: %d' % (results.empty(), size, est_remaining)
          item = results.get()
          results.task_done()
          current += 1
          if current % batch_size == 0 or total_work - current &lt; batch_size:
              if pool is not None and est_remaining == 0 and size/num_procs &gt; batch_size:
                  pool.apply_async(do_work2, args=(current, item, True))
              else:
                  do_work2(current,item, False)
              if current % 1000 == 0:
                  print 'Queue size: %d and remaining work: %d' % (size, est_remaining)
  
  def do_work1(i, w, results):
      time.sleep(.05)
      if i % 1000 == 0:
         print 'did work %d: from %d to %d' % (i,w[0], w[-1])
      for j in w:
          #create an increasing amount of work on the queue
          results.put_nowait(range(j*2))
  
  def do_work2(index, item, in_parallel):
      time.sleep(1)
      if index % 50 == 0:
          print 'processed result %d with length %d in parallel %s' % (index, len(item), in_parallel)
  
  if __name__ == "__main__":
      num_workers = 2
      start = datetime.now()
      print 'Start: %s' % start
      amount_work = 4000
      the_work = [i for i in xrange(amount_work)]
      ppool, results = results_producer(the_work, num_workers)
  
      results_consumer(results, len(the_work), num_workers, ppool)
  
      if ppool is not None:
          ppool.close()
          ppool.join()
      print 'Took: %s time' % (datetime.now() - start)
  </code></pre>
  
  <p>And it deadlocks on the results.put_nowait call from do_work1 even though the queue is empty! Sometimes the code is able to put all the work on the queue but the results.get call from results_consumer blocks since it is apparently empty even though the work has not been consumed yet. </p>
  
  <p>Additionally, I checked the programming guidelines: <a href="https://docs.python.org/2/library/multiprocessing.html" rel="nofollow noreferrer">https://docs.python.org/2/library/multiprocessing.html</a> and believe the above code conforms to it. Lastly the problem in this post: <a href="https://stackoverflow.com/questions/3121622/python-multiprocessing-queue-deadlocks-on-put-and-get">Python multiprocessing.Queue deadlocks on put and get</a> seems very similar and claims to be solved on Windows (I'm running this on Windows 8.1) however the above code doesn't block due to the parent process attempting to join the child process since the logic is similar to the suggested answer. Any suggestions about the cause of the deadlock and how to fix it? Also in general, what is the best way to enable multiple producers to provide results for a consumer to process in python?</p>