title: Python multiprocess change class instance in place
body: <p>I have a list of class instances, and I want to call the same instance method in parallel, use pathos to be able to pickle instance method, The true problem is when I want to change/add an attribute to the instances, it doesn't work, I think this is because the pickling to sub-process is a deep-copy of the inputs. Anyone has any idea how to solve this? I don't want to change the way of writing the instance method ( such as return a value and put it together later). </p>
  
  <pre><code>from joblib import Parallel, delayed
  import pathos.multiprocessing as mp
  # import multiprocessing as mp 
  import random
  import os
  
  pool = mp.Pool(mp.cpu_count())
  
  class Person(object):
      def __init__(self, name):
          self.name = name
  
      def print_name(self, num):
          self.num = num
          print "worker {}, person name {}, received int {}".format(os.getpid(), self.name, self.num)
  
  
  people = [Person('a'),
            Person('b'),
            Person('c'),
            Person('d'),
            Person('e'),
            Person('f'),
            Person('g'),
            Person('h')]
  
  
  for i, per in enumerate(people):
      pool.apply_async(Person.print_name, (per, i) )
  
  pool.close()
  pool.join()
  print 'their number'
  for per in people:
      print per.num
  </code></pre>
  
  <p>This is the output, the num attribute is not found, I think it is because the change is made on those copies.</p>
  
  <pre><code>In [1]: run delme.py
  worker 13981, person name a, random int 0
  worker 13982, person name b, random int 1
  worker 13983, person name c, random int 2
  worker 13984, person name d, random int 3
  worker 13985, person name e, random int 4
  worker 13986, person name f, random int 5
  worker 13987, person name g, random int 6
  worker 13988, person name h, random int 7
  their number
  ---------------------------------------------------------------------------
  AttributeError                            Traceback (most recent call last)
  /chimerahomes/wenhoujx/brain_project/network_analysis/delme.py in &lt;module&gt;()
       39 print 'their number'
       40 for per in people:
  ---&gt; 41     print per.num
  
  AttributeError: 'Person' object has no attribute 'num'
  </code></pre>
  
  <hr>
  
  <p>following suggest in the comments, I try to return self from the child-process, but it seems a pathos bug that the returned self is NOT its original type. See the following code: </p>
  
  <pre><code>import pickle
  # from joblib import Parallel, delayed
  import pathos.multiprocessing as mp
  # import multiprocessing as mp 
  import random
  import os
  
  pool = mp.Pool(mp.cpu_count())
  
  class Person(object):
      def __init__(self, name):
          self.name = name
  
      def print_name(self, num):
          self.num = num
          print "worker {}, person name {}, received int {}".format(os.getpid(), self.name, self.num)
          # return itself and put everything together
          return self
  
  
  
  people = [Person('a'),
            Person('b'),
            Person('c'),
            Person('d'),
            Person('e'),
            Person('f'),
            Person('g'),
            Person('h')]
  
  # Parallel(n_jobs=-1)(delayed(Person.print_name)(per) for per in people)
  
  res = []
  for i, per in enumerate(people):
      res.append(pool.apply_async(Person.print_name, (per, i) ))
  
  pool.close()
  pool.join()
  people = [rr.get() for rr in res]
  
  
  print 'their number'
  for per in people:
      print per.num
  
  print isinstance(people[0], Person)
  </code></pre>
  
  <p>and this is the output: </p>
  
  <pre><code>In [1]: run delme.py
  worker 29963, person name a, received int 0
  worker 29962, person name b, received int 1
  worker 29964, person name c, received int 2
  worker 29962, person name d, received int 3
  worker 29966, person name e, received int 4
  worker 29967, person name f, received int 5
  worker 29966, person name g, received int 6
  worker 29967, person name h, received int 7
  their number
  0
  1
  2
  3
  4
  5
  6
  7
  False
  </code></pre>
  
  <p>I use the default multiprocessing package, and it has no such problem.</p>