title: Python gevent gracefully kill workers
body: <p>I've been digging through our code base for the past week, and I can't really seem to figure it out. Basically, I have a scheduler that spawns a bunch of workers, and I'd like to introduce a feature that, upon receival of a signal (say SIGINT, or SIGTERM) the workers have a grace period (say 60 seconds) to finish their work, and then die. Should all workers finish their work prior to the elapse of the grace period, the process should timeout then and there, and die. To give you an idea --</p>
  
  <p>In the scheduler --</p>
  
  <pre><code>.
  .
  .
  gevent.joinall(map(lambda t: gevent.spawn(t), get_workers()))
  </code></pre>
  
  <p>And in the workers --</p>
  
  <pre><code>def activate(termination_grace_period=60):
      global CURRENT
      global GRACE_PERIOD
      GRACE_PERIOD = termination_grace_period
      CURRENT = gevent.getcurrent()
      signal.signal(signal.SIGINT, die)
  
  def die(signal=None, stack_trace=None):
      global GRACE_PERIOD
      global TASKS
      try:
          with gevent.timeout.Timeout(GRACE_PERIOD) as timeout:
              while TASKS:
                  gevent.idle()
          exit(0)
      except (gevent.timeout.Timeout, gevent.hub.BlockingSwitchOutError):
          exit('exit 1')
      except Exception as e:
          print 'e was', repr(e)
          exit('Machiatto')
  </code></pre>
  
  <p>And so in the scheduler I'll import the <code>activate()</code> method from the worker, and call it before I spawn the workers.
  I'm running into all the exceptions you see in the <code>except</code> block in <code>die()</code>, and the process certainly doesn't timeout; instead, it will throw an exception, then die. The main process is of course the <code>scheduler</code>, which I call from the shell, say, with all the relevant parameters (the grace period, in particular), and so I don't exclude the possibility that the <code>die()</code> handler should live in the <code>scheduler</code> to begin with, since all signal handlers get executed in the main thread anyway, and if I manage to kill the process, all of it's child threads would die anyway.</p>
  
  <p>If anyone has any insight into how I could wire this logic up properly, it would be well appreciated.</p>
  
  <p>Cheers</p>