title: Parallelizing memcpy with OpenMP
body: <p>I need to make copies of two arrays and I'm trying to parallelize the code. I have tried two different ways, but I can't figure out why one is faster than the other.</p>
  
  <p>The first method copies calls <code>memcpy</code> for each array in a separate <code>omp section</code>. Running this with 16 threads takes 0.063 sec.</p>
  
  <pre><code>void copy_uv(void *u, int u_sz, void *v, int v_sz, void *u_copy, void *v_copy) {
      #pragma omp parallel sections
      {
          #pragma omp section
          {
              memcpy(u_copy, u, u_sz);
          }
          #pragma omp section
          {
              memcpy(v_copy, v, v_sz);
          }
      }
  }
  </code></pre>
  
  <p>The second method separates the arrays in blocks depending on the number of threads, and each thread copies its own block. The block sizes are a multiple of 64 bytes (the cache line size) to avoid false sharing. The arrays also were allocated with <code>mkl_malloc(sz, 64)</code> if relevant. This takes 0.135 seconds with 16 threads.</p>
  
  <pre><code>void copy_uv(double *u, int u_sz, double *v, int v_sz, double *u_copy, double *v_copy) {
      int threads = omp_get_num_threads();
      int u_blocksz = ceil((float)u_sz / threads);
      u_blocksz = u_blocksz - (u_blocksz % 64);
      int v_blocksz = ceil((float)v_sz / threads);
      v_blocksz = v_blocksz - (v_blocksz % 64);
  
      int i,j;
      #pragma omp parallel
      {
          #pragma omp for nowait
          for (i = 0; i &lt; threads; ++i) {
              int index_i = i * u_blocksz;
              if (i == threads-1)
                  memcpy(u_copy+index_i, u+index_i, u_sz - index_i);
              else
                  memcpy(u_copy+index_i, u+index_i, u_blocksz);
          }
          #pragma omp for nowait
          for (j = 0; j &lt; threads; ++j) {
              int index_j = j * v_blocksz;
              if (j == threads-1)
                  memcpy(v_copy+index_j, v+index_j, v_sz - index_j);
              else
                  memcpy(v_copy+index_j, v+index_j, v_blocksz);
          }
      }
  }
  </code></pre>
  
  <p>It's true that in this case the last thread might have more memory to copy and might take longer than the others, but even if I give it the same amount of memory to copy and ignore the remaining bytes at the end of the array, the time doesn't change.</p>
  
  <p>The results change little if run with fewer threads. With 4 threads, 0.075s and 0.145s. Why does this happen? The first method uses only two threads, and the second uses all of them, so why is it slower? Is the first method better, or am I just doing something wrong with the second method?</p>