title: How to control memory while using Keras with tensorflow backend?
body: <p>I have created a wrapper class which initializes a <code>keras.models.Sequential</code> model and has a couple of methods for starting the training process and monitoring the progress. I instantiate this class in my <code>main</code> file and perform the training process. Fairly mundane stuff.</p>
  
  <p>My <strong>question</strong> is:</p>
  
  <p>How to free all the GPU memory allocated by <code>tensorflow</code>. I tried the following with no luck:</p>
  
  <pre><code>import keras.backend.tensorflow_backend as K
  with K.get_session() as sess:
      K.set_session(sess)
      import tensorflow as tf
      from neural_net import NeuralNet
      with tf.device('/gpu:0'):
          nn = NeuralNet('config', train_db_path, test_db_path)
          nn.train(1000, 1)
          print 'Done'
      K._SESSION.close()
      K.set_session(None)
  </code></pre>
  
  <p>Even after the session has been closed and reset to <code>None</code>, <code>nvidia-smi</code> does not reflect any reduction in memory usage. Any ideas?</p>
  
  <p><strong>Idea</strong></p>
  
  <p>Would it be meaningful to add a <code>__exit__</code> method to my <code>NeuralNet</code> class and instantiate it as:</p>
  
  <pre><code>with NeuralNet() as nn:
      nn.train(1000, 1)
  </code></pre>
  
  <p>How should I free up the resources of the keras model in this method?</p>
  
  <p><strong>Test environment</strong></p>
  
  <p>I'm using iPython Notebook on an Ubuntu 14.04 with 3 GTX 960 GPUs.</p>
  
  <p>Reference:</p>
  
  <ol>
  <li><a href="https://github.com/fchollet/keras/issues/2102" rel="noreferrer">https://github.com/fchollet/keras/issues/2102</a></li>
  <li><a href="https://groups.google.com/forum/#!topic/keras-users/MFUEY9P1sc8" rel="noreferrer">https://groups.google.com/forum/#!topic/keras-users/MFUEY9P1sc8</a></li>
  </ol>