title: Multiprocessing slower than serial processing in Windows (but not in Linux)
body: <p>I'm trying to parallelize a <code>for loop</code> to speed-up my code, since the loop processing operations are all independent. Following online tutorials, it seems the standard <code>multiprocessing</code> library in Python is a good start, and I've got this working for basic examples.</p>
  
  <p>However, for my actual use case, I find that parallel processing (using a dual core machine) is actually a little (&lt;5%) slower, when run on Windows. Running the same code on Linux, however, results in a parallel processing speed-up of ~25%, compared to serial execution.</p>
  
  <p>From the docs, I believe this may relate to Window's lack of fork() function, which means the process needs to be initialised fresh each time. However, I don't fully understand this and wonder if anyone can confirm this please?</p>
  
  <p>Particularly,</p>
  
  <p>--> Does this mean that all code in the calling python file gets run for each parallel process on Windows, even initialising classes and importing packages? </p>
  
  <p>--> If so, can this be avoided by somehow passing a copy (e.g. using deepcopy) of the class into the new processes?</p>
  
  <p>--> Are there any tips / other strategies for efficient parallelisation of code design for both unix and windows.</p>
  
  <p>My exact code is long and uses many files, so I have created a pseucode-style example structure which hopefully shows the issue.</p>
  
  <pre><code># Imports
  from my_package import MyClass
  imports many other packages / functions
  
  # Initialization (instantiate class and call slow functions that get it ready for processing)
  my_class = Class()
  my_class.set_up(input1=1, input2=2)
  
  # Define main processing function to be used in loop
  def calculation(_input_data):
      # Perform some functions on _input_data
      ......
      # Call method of instantiate class to act on data
      return my_class.class_func(_input_data)
  
  input_data = np.linspace(0, 1, 50)
  output_data = np.zeros_like(input_data)
  
  # For Loop (SERIAL implementation)
  for i, x in enumerate(input_data):
      output_data[i] = calculation(x)
  
  # PARALLEL implementation (this doesn't work well!)
  with multiprocessing.Pool(processes=4) as pool:
      results = pool.map_async(calculation, input_data)
      results.wait()
  output_data = results.get()
  </code></pre>
  
  <p>EDIT: I do not believe the question is a duplicate of the one suggested, since this relates to a difference in Windows and Linunx, which is not mentioned at all in the suggested duplicate question.</p>