title: XGBoost GPU out of memory
body: <p>I am trying to implement a grid search using XGBoost and the Hyperopt library. But I run into the problem shown in the figure: at the 213th configuration, an out of memory error appears. Since my dataset is not very large, I doubt that it is an overload problem due to the data and not even about the parameters of the model on which I grid search. This is because the previous configurations also have more features or training points but the training has not stalled.</p>
  <p><a href="https://i.stack.imgur.com/VoXEZ.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/VoXEZ.png" alt="enter image description here" /></a></p>
  <pre><code>terminate called after throwing an instance of 'thrust::system::system_error'
    what():  parallel_for failed: out of memory
  Aborted (core dumped)
  </code></pre>
  <p>I think maybe it could be a problem related to an accumulation of GPU memory due to the various configurations tested and therefore it is necessary to release it from time to time. But I don't find anything about it.</p>
  <p>Tell me yours, thank you.</p>