title: Python multiprocessing behavior with the number of processes larger than the cores
body: <p>I am trying to use Python's <a href="https://docs.python.org/3.7/library/multiprocessing.html" rel="nofollow noreferrer">multiprocessing</a> library and would like to understand its behavior with different numbers of processes.</p>
  
  <p>I hypothesized that there is no benefit from setting the processes larger than the number of cores.  Contrary to the hypothesis, the experiment code below says that the computation time decreases even with processes above the number of cores (in my case, 4).  </p>
  
  <p>Can someone explain what is going on behind the scene and give some practical guide on how to set the number of processes?</p>
  
  <pre><code>from multiprocessing import Pool, cpu_count
  import time
  from datetime import datetime
  </code></pre>
  
  <p>My CPU counts.</p>
  
  <pre><code>cpu_count()
  # 4
  </code></pre>
  
  <p>An experimental task, which takes approximately 0.5 second.</p>
  
  <pre><code>def f(x):
      time.sleep(0.5)
      return x*x
  
  def execute_time(processes):
      t1 = datetime.now()
      with Pool(processes) as p:
          p.map(f, list(range(36)))
      t2 = datetime.now()
      return t2 - t1
  
  for p in range(1, 25):
      t = execute_time(p)
      print(p, ":", t)
  </code></pre>
  
  <p>Yielded:</p>
  
  <pre><code># 1 : 0:00:18.065411
  # 2 : 0:00:10.051516
  # 3 : 0:00:06.057016
  # 4 : 0:00:04.562439
  # 5 : 0:00:04.069810
  # 6 : 0:00:03.173502
  # 7 : 0:00:03.065977
  # 8 : 0:00:03.082625
  # 9 : 0:00:02.092880
  # 10 : 0:00:02.090963
  # 11 : 0:00:02.061613
  # 12 : 0:00:01.704716
  # 13 : 0:00:01.704880
  # 14 : 0:00:01.615440
  # 15 : 0:00:01.625117
  # 16 : 0:00:01.621259
  # 17 : 0:00:01.639741
  # 18 : 0:00:01.236108
  # 19 : 0:00:01.250113
  # 20 : 0:00:01.255697
  # 21 : 0:00:01.253459
  # 22 : 0:00:01.260632
  # 23 : 0:00:01.262124
  # 24 : 0:00:01.247772
  </code></pre>
  
  <p>It makes sense that the function takes 18 seconds with one process (36 * 0.5 sec = 18 sec).  So does the case with four processes (18 sec / 4 = 4.5 sec).  But I am surprised that the computation time decreases with larger number of processes.</p>