title: CUDA - the launch timed out and was terminated - Ubuntu and no display
body: <p>I am using a workstation containing 4 GeForce GTX Titan black cards for CUDA development. I am working on Ubuntu 12.04.5 and none of these GPUs are used for display. I notice using cudaGetDeviceProperties that kernel execution timeout is enabled. Does this apply when I am not on Windows and not using a display? </p>
  
  <p>I put the following code to test this in one of my kernels which normally runs fine: </p>
  
  <pre><code>__global__ void update1(double *alpha_out, const double *sDotZ, const double *rho, double, *minusAlpha_out, clock_t *global_now)
  {
      clock_t start = clock();
      clock_t now;
  
      for (;;) {
          now = clock();
          clock_t cycles = now &gt; start ? now - start : now + (0xffffffff - start);
          if (cycles &gt;= 50000000000) {
              break;
          }
      }
      *global_now = now;
  }
  </code></pre>
  
  <p>The kernel launch looks like: </p>
  
  <pre><code>update1&lt;&lt;&lt;1, 1&gt;&gt;&gt;(d_alpha + idx, d_tmp, d_rho + idx, d_tmp, global_now);
  CudaCheckError();
  cudaDeviceSynchronize();
  </code></pre>
  
  <p>For a large enough number of cycles waiting, I see the error: </p>
  
  <pre><code>CudaCheckError() with sync failed at /home/.../xxx.cu:295: 
  the launch timed out and was terminated
  </code></pre>
  
  <p>It runs fine for a small number of cycles. If I run this same code on a Tesla K20m GPU with kernel execution timeout disabled, I do not see this error and the program runs as normal. If I see this error, does it definitely mean I am hitting the kernel time limit that appears to be enabled or could there be something else wrong with my code? All mentions of this problem seem to be by people using Windows or also using their card for display so how is it possible I am seeing this error? </p>