title: slow sparse matrix vector product (CSR) using open mp
body: <p>I am trying to speed up a sparse matrix-vector product using open mp, the code is as follows:</p>
  
  <pre><code>void zAx(double * z, double * data, long * colind, long * row_ptr, double * x, int M){
  
  long i, j, ckey;
  int chunk = 1000;
  //int * counts[8]={0};
  #pragma omp parallel num_threads(8)
  { 
    #pragma omp for private(ckey,j,i) schedule(static,chunk)
    for (i=0; i&lt;M; i++ ){ 
      z[i]=0;
      for (ckey=row_ptr[i]; ckey&lt;row_ptr[i+1]; ckey++) {
        j = colind[ckey];
        z[i] += data[ckey]*x[j];
      }              
    }
  }
  }
  </code></pre>
  
  <p>Now, this code runs fine, and produces the correct result, it however only gives me a speed up of ~30%. I have checked to see that the threads are all getting about the same number of non-zero elements (they are), and the matrix is fairly large (300,000 x 300,000), so I'm hoping the overhead isn't the only issue. I've also tried running  with different chunk sizes and thread numbers, and I get similar performance.</p>
  
  <p>Is there something else I could try to get a bit of extra speed out of this? Or something I'm obviously doing wrong?</p>
  
  <p>Cheers.</p>
  
  <p>Edit: Just commented out '//int * counts[8]={0}', because it was leftover from counting the work allocation. Not needed</p>
  
  <p>Edit2 (more details):</p>
  
  <p>Okay so I timed a loop of calling this 5000 times and got the average times: </p>
  
  <ul>
  <li>seq: 0.0036 (seconds?)</li>
  <li>2 threads: 0.002613</li>
  <li>4 threads: 0.002308</li>
  <li>8 threads: 0.002384</li>
  </ul>
  
  <p>The matrix is of size: <strong>303544x303544</strong> and has: <strong>2122980</strong> non-zero elements.</p>
  
  <p>With a much smaller matrix 30000x30000 I get times that go more like</p>
  
  <ul>
  <li>seq 0.000303</li>
  <li>8 threads 0.000078</li>
  </ul>
  
  <p>So it seems the large size may be my issue.</p>