title: Best way to wait for queue population python multiprocessing
body: <p>It is first time I play with parallel computing seriously.
  I am using <code>multiprocessing</code> module in python and I am running into this problem:</p>
  <p>A queue consumer run in a different process then queue producer, the former should wait the latter to finish its job, before stop iterating over the queue. Sometimes the consumer is faster then producer and the queue stays empty.
  If I don't put any condition the program won't stop.</p>
  <p>In the sample code I use the wildcard <code>PRODUCER_IS_OVER</code> to example what I need.</p>
  <p>following code sketch the problem:</p>
  <pre><code>def save_data(save_que, file_):
      ### Coroutine instantiation
      PRODUCER_IS_OVER = False
      empty = False
      ### Queue consumer
      while not(empty and PRODUCER_IS_OVER):
          try:
              data = save_que.get()
              print(&quot;saving data&quot;,data)
          except:
              empty = save_que.empty()
              print(empty)
              pass
          #PRODUCER_IS_OVER = get_condition()
      print (&quot;All data saved&quot;)
      return
  
  def get_condition():
      ###NameError: global name 'PRODUCER_IS_OVER' is not defined
      if PRODUCER_IS_OVER:
          return True
      else:
          return False
  
  
  def produce_data(save_que):
      for _ in range(5):
          time.sleep(random.randint(1,5))
          data = random.randint(1,10)
          print(&quot;sending data&quot;, data)
          save_que.put(data)
  
  ### Main function here
  import random
  import time
  from multiprocessing import Queue, Manager, Process
  manager = Manager()
  save_que = manager.Queue()
  file_ = &quot;file&quot;
  save_p    = Process(target= save_data, args=(save_que, file_))
  save_p.start()
  PRODUCER_IS_OVER = False
  produce_data(save_que)
  PRODUCER_IS_OVER = True
  save_p.join()
  </code></pre>
  <p><code>produce_data</code> takes variable time and I want the save_p process to start BEFORE populate the queue, in order to consume the queue while is filled.
  I think there are workaround to communicate when to stop iteration, but I want to know whether exist a proper way to do it.
  I tried both multiprocessing.Pipe and .Lock, but I don't know how implement correctly and efficiently.</p>
  <p>SOLVED: is it the best way?</p>
  <p>following code implement STOPMESSAGE in the Q, works fine, I can refine it with a class, <code>QMsg</code>, in case the language supports only static types.</p>
  <pre><code>def save_data(save_que, file_):
      # Coroutine instantiation
      PRODUCER_IS_OVER = False
      empty = False
      # Queue consumer
      while not(empty and PRODUCER_IS_OVER):
          data = save_que.get()
          empty = save_que.empty()
          print(&quot;saving data&quot;, data)
          if data == &quot;STOP&quot;:
              PRODUCER_IS_OVER = True
      print(&quot;All data saved&quot;)
      return
  
  
  def get_condition():
      # NameError: global name 'PRODUCER_IS_OVER' is not defined
      if PRODUCER_IS_OVER:
          return True
      else:
          return False
  
  
  def produce_data(save_que):
      for _ in range(5):
          time.sleep(random.randint(1, 5))
          data = random.randint(1, 10)
          print(&quot;sending data&quot;, data)
          save_que.put(data)
      save_que.put(&quot;STOP&quot;)
  
  
  # Main function here
  import random
  import time
  from multiprocessing import Queue, Manager, Process
  manager = Manager()
  save_que = manager.Queue()
  file_ = &quot;file&quot;
  save_p = Process(target=save_data, args=(save_que, file_))
  save_p.start()
  PRODUCER_IS_OVER = False
  produce_data(save_que)
  PRODUCER_IS_OVER = True
  save_p.join()
  </code></pre>
  <p>But this cannot work in case the queue is produced by several separated process: who is going to send the ALT message in that case?</p>
  <p>another solution is to store the processes index in a list and execute:</p>
  <pre><code>def some_alive():
      for p in processes:
          if p.is_alive():
              return True
      return False
  </code></pre>
  <p>But <code>multiprocessing</code> supports <code>.is_alive</code> method only in the parent process, which is limiting in my case.</p>