title: How to pass a function with more than one argument to python concurrent.futures.ProcessPoolExecutor.map()?
body: <p>I would like <code>concurrent.futures.ProcessPoolExecutor.map()</code> to call a function consisting of 2 or more arguments. In the example below, I have resorted to using a <code>lambda</code> function and defining <code>ref</code> as an array of equal size to <code>numberlist</code> with an identical value. </p>
  
  <p><strong>1st Question:</strong> Is there a better way of doing this? In the case where the size of numberlist can be million to billion elements in size, hence ref size would have to follow numberlist, this approach unnecessarily takes up precious memory, which I would like to avoid. I did this because I read the <code>map</code> function will terminate its mapping until the shortest array end is reach. </p>
  
  <pre><code>import concurrent.futures as cf
  
  nmax = 10
  numberlist = range(nmax)
  ref = [5, 5, 5, 5, 5, 5, 5, 5, 5, 5]
  workers = 3
  
  
  def _findmatch(listnumber, ref):    
      print('def _findmatch(listnumber, ref):')
      x=''
      listnumber=str(listnumber)
      ref = str(ref)
      print('listnumber = {0} and ref = {1}'.format(listnumber, ref))
      if ref in listnumber:
          x = listnumber
      print('x = {0}'.format(x))
      return x 
  
  a = map(lambda x, y: _findmatch(x, y), numberlist, ref)
  for n in a:
      print(n)
      if str(ref[0]) in n:
          print('match')
  
  with cf.ProcessPoolExecutor(max_workers=workers) as executor:
      #for n in executor.map(_findmatch, numberlist):
      for n in executor.map(lambda x, y: _findmatch(x, ref), numberlist, ref):
          print(type(n))
          print(n)
          if str(ref[0]) in n:
              print('match')
  </code></pre>
  
  <p>Running the code above, I found that the <code>map</code> function was able to achieve my desired outcome. However, when I transferred the same terms to  concurrent.futures.ProcessPoolExecutor.map(), python3.5 failed with this error:</p>
  
  <pre><code>Traceback (most recent call last):
    File "/usr/lib/python3.5/multiprocessing/queues.py", line 241, in _feed
      obj = ForkingPickler.dumps(obj)
    File "/usr/lib/python3.5/multiprocessing/reduction.py", line 50, in dumps
      cls(buf, protocol).dump(obj)
  _pickle.PicklingError: Can't pickle &lt;function &lt;lambda&gt; at 0x7fd2a14db0d0&gt;: attribute lookup &lt;lambda&gt; on __main__ failed
  </code></pre>
  
  <p><strong>Question 2</strong>: Why did this error occur and how do I get  concurrent.futures.ProcessPoolExecutor.map() to call a function with more than 1 argument?</p>