title: Python multiprocessing script dies after several hours of running
body: <p>Am having a python3.4 multiprocessing script that runs ok and then dies out on me. For example, it runs for about 4 hours then when I come back to it I find its no longer processing. </p>
  
  <p>As opposed to earlier, Top now shows a gazillion processes running in the background. I ordered them and found a bunch of python3.4 processes.  </p>
  
  <pre><code>  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
  13322 root      20   0  888716 316456   3272 S   0.0  7.8   0:27.09 python3.4
  13325 root      20   0  888480 316212   3272 S   0.0  7.8   0:26.54 python3.4
  13327 root      20   0  873136 300836   3272 S   0.0  7.4   0:14.83 python3.4
  13309 root      20   0  651924 299896   3244 S   0.0  7.4   0:23.38 python3.4
  13305 root      20   0  651924 299888   3244 S   0.0  7.4   0:22.17 python3.4
  13287 root      20   0 1055324  46800   6268 S   0.0  1.2   0:03.97 python3.4
  13414 root      20   0  834128  41344   3264 S   2.3  1.0   0:01.92 python3.4
  13415 root      20   0  834128  41344   3264 S   2.0  1.0   0:01.82 python3.4
  13416 root      20   0  834128  41344   3264 S   2.0  1.0   0:01.85 python3.4
  13417 root      20   0  834128  41344   3264 S   2.0  1.0   0:01.88 python3.4
  13418 root      20   0  834128  41344   3264 S   2.3  1.0   0:01.86 python3.4
  13419 root      20   0  834128  41344   3264 S   2.7  1.0   0:01.89 python3.4
  13420 root      20   0  834128  41344   3264 S   2.0  1.0   0:01.83 python3.4
  #About 60 others truncated for brevity
  13293 root      20   0  165356  31632   3260 S   0.0  0.8   0:02.46 python3.4
  13291 root      20   0  165364  31592   3256 S   0.0  0.8   0:02.37 python3.4
  13295 root      20   0  165356  31584   3256 S   0.0  0.8   0:02.44 python3.4
  </code></pre>
  
  <p>Below is the setup code to start the processing. I've observed using top that upon each subsequent call to <code>process_score</code>, existing python3.4 processes remain and a new bunch of 8 come up. Eventually, I think the processes build up and eat up my resources and cause the main script to "freeze". </p>
  
  <p>Its my thinking that part of the magic of <code>pool.map</code> is terminating child processes on exit and I don't have to keep track and terminate them myself? As such, how do I ensure that the child processes are terminated? Searching in stackoverflow doesn't give much information on terminating child processes while using pool.map</p>
  
  <pre><code>While True
      id_list = get_student_ids()
      if id_list == []:
          break
      else:
          #Multiprocessing starts here:
          num_of_consumers = multiprocessing.cpu_count()
          pool = multiprocessing.Pool(num_of_consumers)
          col_list_insert = pool.map(student_score.process_score, id_list)
  </code></pre>