title: python multiprocessing.Pool kill *specific* long running or hung process
body: <p>I need to execute a pool of many parallel database connections and queries. I would like to use a multiprocessing.Pool or concurrent.futures ProcessPoolExecutor. Python 2.7.5</p>
  
  <p>In some cases, query requests take too long or will never finish (hung/zombie process). I would like to kill the <em>specific</em> process from the multiprocessing.Pool or concurrent.futures ProcessPoolExecutor that has timed out.</p>
  
  <p>Here is an example of how to kill/re-spawn the entire process pool, but ideally I would minimize that CPU thrashing since I only want to kill a specific long running process that has not returned data after timeout seconds.</p>
  
  <p>For some reason the code below does not seem to be able to terminate/join the process Pool after all results are returned and completed. It may have to do with killing worker processes when a timeout occurs, however the Pool creates new workers when they are killed and results are as expected.</p>
  
  <pre><code>from multiprocessing import Pool
  import time
  import numpy as np
  from threading import Timer
  import thread, time, sys
  
  def f(x):
      time.sleep(x)
      return x
  
  if __name__ == '__main__':
      pool = Pool(processes=4, maxtasksperchild=4)
  
      results = [(x, pool.apply_async(f, (x,))) for x in np.random.randint(10, size=10).tolist()]
  
      while results:
          try:
              x, result = results.pop(0)
              start = time.time()
              print result.get(timeout=5), '%d done in %f Seconds!' % (x, time.time()-start)
  
          except Exception as e:
              print str(e)
              print '%d Timeout Exception! in %f' % (x, time.time()-start)
              for p in pool._pool:
                  if p.exitcode is None:
                      p.terminate()
  
      pool.terminate()
      pool.join()
  </code></pre>