title: How do I feed an infinite generator to eventlet (or gevent)?
body: <p>The docs of both eventlet and gevent have several examples on how to asyncronously spawn IO tasks and get the results latter.
  But so far, all the examples where a value should be returned from the async call,I allways find a blocking call after all the calls to <code>spawn()</code>. Either <code>join()</code>, <code>joinall()</code>, <code>wait()</code>, <code>waitall()</code>.
  This assumes that calling the functions that use IO is immediate and we can jump right into the point where we are waiting for the results.</p>
  
  <p>But in my case I want to get the jobs from a generator that can be slow and or arbitrarily large or even infinite.</p>
  
  <p>I obviously can't do this</p>
  
  <pre><code>pile = eventlet.GreenPile(pool)
  for url in mybiggenerator():
      pile.spawn(fetch_title, url)
  titles = '\n'.join(pile)
  </code></pre>
  
  <p>because <code>mybiggenerator()</code> can take a long time before it is exhausted. So I have to start consuming the results while I am still spawning async calls.</p>
  
  <p>This is probably usually done with resource to queues, but I'm not really sure how. Say I create a queue to hold jobs, push a bunch of jobs from a greenlet called P and pop them from another greenlet C.
  When in C, if I find that the queue is empty, how do I know if P has pushed every job it had to push or if it is just in the middle of an iteration?</p>
  
  <p>Alternativey,Eventlet allows me to loop through a <code>pile</code> to get the return values, but can I start doing this without having spawn all the jobs I have to spawn? How? This would be a simpler alternative.</p>