title: CUDA vs Intel AVX / SSE vector sum performance questions
body: <p>First of all I'm new to CUDA and I'm trying to learn, so maybe I'm doing something wrong. I wanted to compare the CUDA performance vs the equivalent function implemented with Intel intrinsics expecting that CUDA will yield a better result.</p>
  
  <p>To my surprise though, thats not what I'm seeing. My function is extremely simple, I just add two vectors and store the results in a third one. My CUDA code is as basic as it gets, in the setup function I have :</p>
  
  <pre><code>void cudaAddVectors(float* vectorA, float* vectorB, float* sum, int numElements)
  {
  //
  // Allocate the memory on the device
  //
  float* dvA;
  float* dvB;
  float* dvC;
  
  cudaMalloc((void**)&amp;dvA, numElements * sizeof(float));
  cudaMalloc((void**)&amp;dvB, numElements * sizeof(float));
  cudaMalloc((void**)&amp;dvC, numElements * sizeof(float));
  
  //
  // Copy the host vectors to device vectors
  //
  cudaMemcpy(dvA, vectorA, numElements * sizeof(float), cudaMemcpyHostToDevice);
  cudaMemcpy(dvB, vectorB, numElements * sizeof(float), cudaMemcpyHostToDevice);
  
  //
  // Perform the sum on the device and time it
  //
  deviceSumLink(dvA, dvB, dvC, numElements);
  
  //
  // Now get the results back to the host
  //
  cudaMemcpy(sum, dvC, numElements * sizeof(float), cudaMemcpyDeviceToHost);
  
  // Cleanup and go home
  cudaFree(dvA);
  cudaFree(dvB);
  cudaFree(dvC);
  </code></pre>
  
  <p>}</p>
  
  <p>then the device code is run either with blocks or threads, like so:</p>
  
  <pre><code>void deviceSumLink(float* a, float* b, float* c, int numElements)
  {
      //deviceSum&lt;&lt;&lt;numElements, 1&gt;&gt;&gt;(a,b,c);
      deviceSumThreads&lt;&lt;&lt;1, numElements&gt;&gt;&gt;(a,b,c);
  }
  </code></pre>
  
  <p>And the actual code running on the device:</p>
  
  <pre><code>__global__ void deviceSum(float* a, float* b, float* c)
  {
      int index = blockIdx.x;
      c[index] = a[index] + b[index];
  }
  </code></pre>
  
  <p>or </p>
  
  <pre><code>__global__ void deviceSumThreads(float* a, float* b, float* c)
  {
      int index = threadIdx.x;
      c[index] = a[index] + b[index];
  }
  </code></pre>
  
  <p>I timed the Intel version of this and the CUDA summing different size vectors and verifying that both produced accurate results. For the CUDA calls, I'm timing only the deviceSumLink call, not the memory setup and everything, but regardless of the method of invoking the kernels, the Intel intrinsics version (using 8-element arrays) is just smoking the CUDA out of the water. Basically, the Intel SIMD version of the function is something like 10x faster!</p>
  
  <p>I did not expect this, so I attribute this to me being a complete newbie in CUDA. So what am I doing wrong? I thought CUDA was supposed to be much faster in those kinds of things, I think I must not be using it right or something.</p>
  
  <p>If you have some insight, I'd appreciate the comments!</p>
  
  <p>Thx!</p>