title: Handle several chained-function with python multiprocessing (pool)
body: <p>Coding is pretty new for me, so don't mind me if what I'm asking is quite easy or idiotic.</p>
  
  <p>I wrote a python script for doing some obscure stuff, long story short, I'm trying to compare some strings from similar objects from 4 different files. </p>
  
  <p>First I need to fetch the objects in each file, then transform the part of the object which I'm interested in, in a format I can compare them, then compare, and finally write the results in a file. </p>
  
  <p>Unfortunately, it is super slow. I know where my bottleneck steps are (the object transformation part, function2()), I would like to improve the speed of my program with some multiprocessing.</p>
  
  <p>I tried several things, the only one I managed to get working is the pool function. </p>
  
  <p>So I made a big function summarising all my steps and try to make it works with pool workers.</p>
  
  <pre><code>import multiprocessing as mp
  
  
  def function1(name_object,input):
  
      fetch the objects in input
      return objects
  
  def function2(objects) :
  
      transforming the objects into strings
      return strings
  
  def function3(strings,output) :
  
      compare each position of the strings
      outptut.write(results)
  
  def function_finale(name_object,input,output) :
      objects = function1(name_object,input)
      strings = function2(objects)
      results = function3(strings,output)
  
  if __name__ == '__main__':
  
      pool = mp.Pool(processes=4)
  
      [pool.apply_async(combine, args=(object,file)) for object in file]
  </code></pre>
  
  <p>(Un)surprisingly the pool workers terminate the function after the first subfunction. I think I can do parallel programming, because my objects can be handled separately, asynchronously in any order, but the order of my functions must be followed once an object is fetched.</p>
  
  <p>Basically, I would like to handle several objects in the same time in my loop with several processes! It should be possible, right?</p>
  
  <p>I got some constraint, for example, I cannot stock everything in the dictionary because I got 300k+ objects which are pretty heavy (indeed, I'm doing this because of memory issues first).</p>
  
  <p>Any advice?</p>
  
  <p>Cheers.</p>