title: Can OpenMP 4 runs target regions in parallel?
body: <p>Reading some tutorials from OpenMP 4, I found that <code>target</code> regions can participate in the same dependency graph of CPU tasks, using the <code>depend</code> clause.</p>
  
  <p>When programming OpenMP tasks, we know they can be run concurrently. But is this possible on GPUs? Can a GPU run multiple <code>target</code> regions simultaneously?</p>
  
  <p>I tried with this code:</p>
  
  <pre class="lang-c prettyprint-override"><code>#include &lt;omp.h&gt;
  #include &lt;stdio.h&gt;
  
  int main() {
    int i;
  
  #pragma omp parallel
  #pragma omp single
    {
  
  #pragma omp task private(i)
  #pragma omp target
      {
        for (i = 0; i &lt; 100; i++)
          printf("1 %d\n", i);
      }
  
  #pragma omp task private(i)
  #pragma omp target
      {
        for (i = 0; i &lt; 100; i++)
          printf("2 %d\n", i);
      }
  
  #pragma omp task private(i)
  #pragma omp target
      {
        for (i = 0; i &lt; 100; i++)
          printf("3 %d\n", i);
      }
    }
  
  #pragma omp taskwait
  }
  </code></pre>
  
  <p>Although the <code>task</code>s are executed in arbitrary order, the <code>target</code> regions are executed atomically, one region at a time.</p>