title: python multiprocessing scheduling task
body: <p>I have 8 CPU core and 200 tasks to done. Each tasks are isolate. There is no need to wait or share the result. I'm looking for a way to run 8 tasks/processes at a time (Maximum) and when one of them finished. The remaining task will automatic start process.</p>
  
  <p>How to know when the child process was done and start a new child process. First I'm trying to use process(multiprocessing) and it's hard to figure out. Then I try to use pool and face with the pickle problem cause I need to use dynamic instantiate.</p>
  
  <p>Edited : Adding my code of Pool</p>
  
  <pre><code>class Collectorparallel():
  
  def fire(self,obj):
      collectorController = Collectorcontroller()
      collectorController.crawlTask(obj)
  
  def start(self):
      log_to_stderr(logging.DEBUG)
      pluginObjectList = []
      for pluginName in self.settingModel.getAllCollectorName():
          name = pluginName.capitalize()
          #Get plugin class and instanitiate object
          module = __import__('plugins.'+pluginName,fromlist=[name])
          pluginClass = getattr(module,name)
          pluginObject = pluginClass()
          pluginObjectList.append(pluginObject)
  
  
  
      pool = Pool(8)
      jobs = pool.map(self.fire,pluginObjectList)
      pool.close()
  
      print pluginObjectList
  </code></pre>
  
  <p>pluginObjectList got something like </p>
  
  <pre><code>[&lt;plugins.name1.Name1 instance at 0x1f54290&gt;, &lt;plugins.name2.Name2 instance at 0x1f54f38&gt;]
  </code></pre>
  
  <p>PicklingError: Can't pickle : attribute lookup <strong>builtin</strong>.instancemethod failed</p>
  
  <p>but the Process version work fine</p>