title: sharing a dictionary and an array among pool of processes
body: <p>I have been trying to create a dictionary that will have a device mac id as key and information corresponding to that mac in a list.Something like this.</p>
  
  <pre><code>{00-00-0A-14-01-06:[['CMTS-51-55_10.20', '10.20.1.1', '342900', 'Cable6/0/0', '110', 'Cable6/0/0-upstream0', '129', 'Cable6/0/0-downstream', '00-00-0A-14-01-06', '10.20.1.6', '11', '1', '1424419744000', '692306', 'SignalingDown', '1', '118800000', '990000', '0', '0', '0', '342900'], 
  ['CMTS-51-55_10.20', '10.20.1.1', '343800', 'Cable6/0/0', '110', 'Cable6/0/0-upstream0', '129', 'Cable6/0/0-downstream', '00-00-0A-14-01-06', '10.20.1.6', '11', '1', '1424420644000', '692306', 'SignalingDown', '1', '118800000', '990000', '0', '0', '0', '343800'], 
  ['CMTS-51-55_10.20', '10.20.1.1', '342900', 'Cable6/0/0', '110', 'Cable6/0/0-upstream0', '129', 'Cable6/0/0-downstream', '00-00-0A-14-01-06', '10.20.1.6', '11', '1', '1424419744000', '377773', 'SignalingUp', '2', '118800000', '990000', '0', '0', '0', '342900']]} 
  </code></pre>
  
  <p>These data values are retrieved from multiple files kept in multiple folder. One folder can have multiple files.</p>
  
  <p>I am giving this list of folder to a pool of processes. So that within one process all the files from one folder get executed.</p>
  
  <p>I am maintaining a local dictionary(collection.defaultdict) to populate it with complete information and then putting that information in shared dictionany(manager.dict) which i am giving as an argument to pool object.</p>
  
  <p>also i am giving a character array to share some template information between the child processes and main process.</p>
  
  <p>I am trying to check the sharing task in the multiprocessing part but i don't appear to get it working.</p>
  
  <p>Kindly someone help me with it.</p>
  
  <pre><code>#!/usr/local/bin/pypy
  
  from multiprocessing import Process
  from multiprocessing import Pool, Manager ,Value, Array
  import collections
  from collections import defaultdict
  import itertools
  import os
  
  def info(title):
      print title
      print 'module name:', __name__
      if hasattr(os, 'getppid'):  # only available on Unix
          print 'parent process:', os.getppid()
      print 'process id:', os.getpid()
  
  def f(template,mydict):
      name = 'bob'
      info('function f')
      resultDeltaArray = collections.defaultdict(list)
      resultDeltaArray['b'].append("hi")
      resultDeltaArray['b'].append("bye")
      resultDeltaArray['c'].append("bye")
      resultDeltaArray['c'].append("bye")
      template = "name"
      print resultDeltaArray
      #print "templaate1", template
      for k,v in resultDeltaArray.viewitems():
          mydict[k] = v
      print 'hello', name
      #mydict = resultDeltaArray
      for k,v in mydict.items():
          print mydict[k]
          #del mydict[k]
  
  if __name__ == '__main__':
      info('main line')
      manager = Manager()
      mydict = manager.dict()
      template = Array('c',50)
      #mydict[''] = []
      #print mydict
      todopool = Pool(2)
      todopool.map_async(f, itertools.repeat(template),itertools.repeat(mydict))
      #print "hi"
      #p = Process(target=f, args=('bob',template,mydict))
      #p.start()
      #p.join()
      print mydict
      mydict.clear()
      print mydict
  
      print "template2", template
  </code></pre>
  
  <p>code is to check the multiprocessing part. It is not the actual implementation.
  In this case it is just getting hung and not doing anything after printing:</p>
  
  <pre><code>main line
  module name: __main__
  parent process: 27301
  process id: 27852
  </code></pre>
  
  <p>and when i try to interrupt the process using ctrl-C it again get stuck after printing </p>
  
  <pre><code>Traceback (most recent call last):
    File "/home/pydev/checkouts/dev/trunk/thirdparty/pypy_2.1/lib-python/2.7/multiprocessing/process.py", line 258, in _bootstrap
    Process PoolWorker-2:
  Traceback (most recent call last):
    File "/home/pydev/checkouts/dev/trunk/thirdparty/pypy_2.1/lib-python    /2.7/multiprocessing/process.py", line 258, in _bootstrap
      self.run()
    File "/home/pydev/checkouts/dev/trunk/thirdparty/pypy_2.1/lib-python /2.7/multiprocessing/process.py", line 114, in run
      self._target(*self._args, **self._kwargs)
    File "/home/pydev/checkouts/dev/trunk/thirdparty/pypy_2.1/lib-python/2.7/multiprocessing/pool.py", line 85, in worker
      self.run()
    File "/home/pydev/checkouts/dev/trunk/thirdparty/pypy_2.1/lib-python/2.7/multiprocessing/process.py", line 114, in run
      self._target(*self._args, **self._kwargs)
    File "/home/pydev/checkouts/dev/trunk/thirdparty/pypy_2.1/lib-python/2.7/multiprocessing/pool.py", line 85, in worker
      task = get()
    File "/home/pydev/checkouts/dev/trunk/thirdparty/pypy_2.1/lib-python/2.7/multiprocessing/queues.py", line 374, in get
      racquire()
  KeyboardInterrupt
      task = get()
    File "/home/pydev/checkouts/dev/trunk/thirdparty/pypy_2.1/lib-python/2.7/multiprocessing/queues.py", line 376, in get
      return recv()
  </code></pre>
  
  <p>Am I using things in a correct way? Does Pool object not allow multiprocessing array or manager.dict as an argument? Is there some other way of doing the same thing?  </p>