title: GPU runs during the dev step out of memory
body: <p>I'm using the <a href="https://github.com/dennybritz/cnn-text-classification-tf" rel="nofollow noreferrer">Denny Britz implementation</a> of <a href="https://arxiv.org/abs/1408.5882" rel="nofollow noreferrer">Yoon Kims CNN</a> for sentiment analysis in slightly modified form (I added the word2vec approach, so that the weight matrix is not calculated from scratch).</p>
  
  <p>With small datasets (like 10MB) it works fine, but if I try to train on datasets of size >50MB (still not very large) my GPU runs out of memory and throws the following error: <a href="http://pastebin.com/AMfYkpXZ" rel="nofollow noreferrer">http://pastebin.com/AMfYkpXZ</a></p>
  
  <p>The GPU is a GeForce GTX 1080 with 8 gb.</p>
  
  <p>I worked out that the error comes from the dev step/evaluation step:</p>
  
  <pre><code>def dev_step(x_batch, y_batch, writer=None):
      """
      Evaluates model on a dev set
      """
      feed_dict = {
          cnn.input_x: x_batch,
          cnn.input_y: y_batch,
          cnn.dropout_keep_prob: 1.0
      }
      step, summaries, loss, accuracy = sess.run(
          [global_step, dev_summary_op, cnn.loss, cnn.accuracy],
          feed_dict)
      time_str = datetime.datetime.now().isoformat()
      print("{}: step {}, loss {:g}, acc {:g}".format(time_str, step, loss, accuracy))
      if writer:
          writer.add_summary(summaries, step)
  </code></pre>
  
  <p>To be precisely, it comes from the <code>sess.run([global_step, dev_summary_op, cnn.loss, cnn.accuracy], feed_dict)</code> line.
  If i comment out the whole dev_step, the training runs without throwing errors. </p>
  
  <p>Do you have an idea why this error occurs and how I can fix it? Thanks in advance!</p>
  
  <p><strong>Edit:</strong>
  The whole code is available at: <a href="https://gist.github.com/pexmar/7b3d074825eeaf5a336fce406d8e9bae" rel="nofollow noreferrer">https://gist.github.com/pexmar/7b3d074825eeaf5a336fce406d8e9bae</a></p>