title: Why I don't see different/unique outputs of my variable value after each atomicAdd?
body: <p>I'm new to CUDA.
  I was trying to implement a trie data structure on GPU but it didn't work. I noticed my atomicAdd isn't working as I expected.
  So I did some experiment with atomicAdd. I wrote this piece of code :</p>
  
  <pre><code>#include &lt;cstdio&gt;
  
  //__device__ int *a; //I also tried the code with using this __device__
                       //variable and allocating it inside kernel instead
                       //using cudaMalloc. Same Result
  
  __global__ void AtomicTestKernel (int*a)
  {
      *a = 0;
      __syncthreads();
      for (int i = 0; i &lt; 2; i++)
      {
          if (threadIdx.x % 2)
          {
              atomicAdd(a, 1);
              printf("threadsIndex = %d\t&amp;\ta : %d\n",threadIdx.x,*a);
          }
          else
          {
              atomicAdd(a, 1);
              printf("threadsIndex = %d\t&amp;\ta : %d\n", threadIdx.x, *a);
          }
      }
  }
  
  int main()
  {
      int * d_a;
      cudaMalloc((void**)&amp;d_a, sizeof(int));
  
      AtomicTestKernel &lt;&lt; &lt;1, 10 &gt;&gt; &gt; (d_a);
  
      cudaDeviceSynchronize();
  
      return 0;
  }
  </code></pre>
  
  <p>correct me where I'm wrong about this code :</p>
  
  <p>1 - according to CUDA's Programming guide : (on atomic Functions) </p>
  
  <blockquote>
    <p>... In other words, no other thread can access this address until the
    operation is complete</p>
  </blockquote>
  
  <p>2 - the <code>int * d_a</code> resides in global memory and so is the kernel's input : <code>int * a</code>
  because its allocated using <code>cudaMalloc</code> (according to this 3 minute video : <a href="https://www.youtube.com/watch?v=19qJjMFKG3g&amp;index=93&amp;list=PLGvfHSgImk4aweyWlhBXNF6XISY3um82_" rel="nofollow noreferrer">Udacity CUDA - Global Memory</a>) and therefore all of threads are seeing the same <code>int * a</code> and not each of them has it's own</p>
  
  <p>3 - In the code as before every <code>printf</code> there is an <code>atomicAdd</code> so I expect each of those <code>printf</code>s has the value of <code>*a</code> different from previous and therefore unique.</p>
  
  <p>BUT in the result I get I see so many same variable of <code>*a</code>
   this is the result I get :</p>
  
  <pre><code>threadsIndex = 0        &amp;       a : 5
  threadsIndex = 2        &amp;       a : 5
  threadsIndex = 4        &amp;       a : 5
  threadsIndex = 6        &amp;       a : 5
  threadsIndex = 8        &amp;       a : 5
  threadsIndex = 1        &amp;       a : 10
  threadsIndex = 3        &amp;       a : 10
  threadsIndex = 5        &amp;       a : 10
  threadsIndex = 7        &amp;       a : 10
  threadsIndex = 9        &amp;       a : 10
  threadsIndex = 0        &amp;       a : 15
  threadsIndex = 2        &amp;       a : 15
  threadsIndex = 4        &amp;       a : 15
  threadsIndex = 6        &amp;       a : 15
  threadsIndex = 8        &amp;       a : 15
  threadsIndex = 1        &amp;       a : 20
  threadsIndex = 3        &amp;       a : 20
  threadsIndex = 5        &amp;       a : 20
  threadsIndex = 7        &amp;       a : 20
  threadsIndex = 9        &amp;       a : 20
  Press any key to continue . . .
  </code></pre>