title: Mutiprocessing in AWS Lambda
body: <p>I was trying to trigger lambdas concurrently using Dynamodb streams with ProcessPoolExecutor. This is the error I received. </p>
  
  <p>The scenario is when a bunch of records(like 1000 records) fall into dynamodb(batch size 10), the streams trigger a lambda. I used ThreadPoolExecutor before and it worked but out of 10 processes in a batch only 5-8 records are processed and remaining are left.Each records takes around 50 seconds to finish. is it the 5 mins limit of AWS lambda that is skipping the other records in ThreadPoolExecutor. Also, using ProcessPoolExecutor will help my problem with ThreadPoolExecutor???</p>
  
  <pre><code>    [Errno 38] Function not implemented: OSError
      Traceback (most recent call last):
      File "/var/task/ycf_calculator.py", line 464, in main
      with ProcessPoolExecutor(max_workers=25) as executor:
      File "/var/lang/lib/python3.6/concurrent/futures/process.py", line 390, in __init__
      EXTRA_QUEUED_CALLS)
      File "/var/lang/lib/python3.6/multiprocessing/context.py", line 102, in Queue
      return Queue(maxsize, ctx=self.get_context())
      File "/var/lang/lib/python3.6/multiprocessing/queues.py", line 42,in __init__
      self._rlock = ctx.Lock()
      File "/var/lang/lib/python3.6/multiprocessing/context.py", line 67, in Lock
      return Lock(ctx=self.get_context())
      File "/var/lang/lib/python3.6/multiprocessing/synchronize.py", line 163, in __init__
      SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)
      File "/var/lang/lib/python3.6/multiprocessing/synchronize.py", line 60, in __init__
      unlink_now)
      OSError: [Errno 38] Function not implemented
  </code></pre>