title: PyCUDA clean-up error, CUDA launch timed out error, on some machines only
body: <p>I have a Python 3 <a href="https://gitlab.com/AmosEgel/smuthi" rel="nofollow noreferrer">program</a> that involves the execution of a <a href="https://gitlab.com/AmosEgel/smuthi/blob/master/smuthi/cuda_sources.py" rel="nofollow noreferrer">cuda kernel</a>. </p>
  
  <p>The code runs fine when I launch it in the following configuration</p>
  
  <ul>
  <li>GeForce GTX 1080 Ti GPU </li>
  <li>Ubuntu 16.04 </li>
  <li>CUDA version 8.0.61 </li>
  <li>NVIDIA driver version 384.111 </li>
  <li>Python version 3.5.2</li>
  <li>PyCUDA version (2017, 1, 1). </li>
  </ul>
  
  <p>However, when using a GeForce GTX 970 on the very same machine, I get this error:</p>
  
  <pre><code>cuMemFree failed: the launch timed out and was terminated
  PyCUDA WARNING: a clean-up operation failed (dead context maybe?)
  </code></pre>
  
  <p>Note that this error does not occur when I call the kernel with a rather small number of threads (i.e. with a small grid dimension at constant threads per block).</p>
  
  <p><a href="http://pycuda.2962900.n2.nabble.com/PyCUDA-PyCUDA-warning-clean-up-operation-errors-on-Kepler-td7575401.html" rel="nofollow noreferrer">In this post</a>, Andreas explains the meaning of that error message: </p>
  
  <blockquote>
    <p>This means your context went away while PyCUDA was still talking to
    it. This will happen most often if you perform some invalid operation
    (such as access out-of-bounds memory in a kernel).</p>
  </blockquote>
  
  <p>In other words, it seems to indicate that something is wrong with the kernel I wrote. However, as the code does not raise an error when launched on the other GPU, I was wondering if other issues can raise the same error, too.</p>
  
  <p>So my questions are: </p>
  
  <ul>
  <li>Can the above error also be caused when running a correctly written kernel in an unfavourable environment? </li>
  <li>Can it be caused by a wrong combination of NVIDIA driver, CUDA version, PyCUDA version and GPU model?</li>
  <li>What do I have in general to consider regarding driver version, CUDA version, PyCUDA version and GPU model to assure that things function properly?</li>
  </ul>
  
  <p>I can understand that many people here are allergic to questions without code and minimal example. I tried to compose a simple example that would reproduce the error, but I couldn't. Kernels that would like double an input argument or so run fine up to the limit of memory errors... So I hope to just get some advice <em>into what direction to look</em> when searching the error.</p>