title: numba - guvectorize barely faster than jit
body: <p>I was trying to parallellize a Monte Carlo simulation that operates on many independent datasets. I found out that numba's parallel guvectorize implementation was barely 30-40% faster than the numba jit implementation.</p>
  
  <p>I found these (<a href="https://stackoverflow.com/questions/31101699/numba-guvectorize-is-very-slow-compared-to-jit">1</a>, <a href="https://stackoverflow.com/questions/35350689/numba-guvectorize-target-parallel-slower-than-target-cpu">2</a>) comparable topics on Stackoverflow, but they do not really answer my question. In the first case, the implementation is slowed down by a fall back to object mode and in the second case the original poster did not properly use guvectorize - none of these problems apply to my code.</p>
  
  <p>To make sure there was no problem with my code, I created this very simple piece of code to compare jit to guvectorize:</p>
  
  <pre><code>import timeit
  import numpy as np
  from numba import jit, guvectorize
  
  #both functions take an (m x n) array as input, compute the row sum, and return the row sums in a (m x 1) array
  
  @guvectorize(["void(float64[:], float64[:])"], "(n) -&gt; ()", target="parallel", nopython=True)
  def row_sum_gu(input, output) :
      output[0] = np.sum(input)
  
  @jit(nopython=True)
  def row_sum_jit(input_array, output_array) :
      m, n = input_array.shape
      for i in range(m) :
          output_array[i] = np.sum(input_array[i,:])
  
  rows = int(64) #broadcasting (= supposed parallellization) dimension for guvectorize
  columns = int(1e6)
  input_array = np.ones((rows, columns))
  output_array = np.zeros((rows))
  output_array2 = np.zeros((rows))
  
  #the first run includes the compile time
  row_sum_jit(input_array, output_array)
  row_sum_gu(input_array, output_array2)
  
  #run each function 100 times and record the time
  print("jit time:", timeit.timeit("row_sum_jit(input_array, output_array)", "from __main__ import row_sum_jit, input_array, output_array", number=100))
  print("guvectorize time:", timeit.timeit("row_sum_gu(input_array, output_array2)", "from __main__ import row_sum_gu, input_array, output_array2", number=100))
  </code></pre>
  
  <p>This gives me the following output (the times do vary a bit):</p>
  
  <pre><code>jit time: 12.04114792868495
  guvectorize time: 5.415564753115177
  </code></pre>
  
  <p>Thus again, the parallel code is barely two times faster (only when the number of rows is an integer multiple of the number of CPU cores, otherwise the performance advantage diminishes) even though it utilizes all cpu cores and the jit code only uses one (verified using htop).</p>
  
  <p>I am running this on a machine with 4x AMD Opteron 6380 CPU (so 64 cores in total), 256 GB of RAM and Red Hat 4.4.7-1 OS.
  I use Anaconda 4.2.0 with Python 3.5.2 and Numba 0.26.0.</p>
  
  <p>How can I further improve the parallel performance or what am I doing wrong?</p>
  
  <p>Thank you for your answers.</p>