title: Tensorflow: How to have saver.save() and .restore() in one module?
body: <p>I have a module called <code>neural.py</code></p>
  
  <p>I initialize the variables in the body.</p>
  
  <pre><code>import tensorflow as tf 
  tf_x = tf.placeholder(tf.float32, [None, length])
  tf_y = tf.placeholder(tf.float32, [None, num_classes])
  ...
  </code></pre>
  
  <p>I save the checkpoint in a function <code>train()</code> after training:</p>
  
  <pre><code>def train():
      ...
      pred = tf.layers.dense(dropout, num_classes, tf.identity) 
      ...
      cross_entropy = tf.losses.softmax_cross_entropy(tf_y, pred)
      ...
      with tf.Session() as sess:
          init = tf.global_variables_initializer()
          sess.run(init)
          saver = tf.train.Saver(tf.trainable_variables())
          for ep in range(epochs):
          ... (training steps)...
      saver.save(sess, "checkpoints/cnn") 
  </code></pre>
  
  <p>I want to also restore and run the network after training in the <code>run()</code> function of this module:</p>
  
  <pre><code>def run():
      # I have tried adding tf.reset_default_graph() here
      # I have also tried with tf.Graph().as_default() as g: and adding (graph=g) in tf.Session()
  
      saver = tf.train.Saver()
  
      with tf.Session() as sess:
          saver.restore(sess, "checkpoints/cnn")
          ... (run network etc)
  </code></pre>
  
  <p>It just doesn't work. It gives me either <code>NotFoundError (see above for traceback): Key beta2_power not found in checkpoint</code> or <code>ValueError: No variables to save</code> if I add <code>tf.reset_default_graph()</code> under <code>run()</code>, as commented above.</p>
  
  <p>However, if I put the exact same code for <code>run()</code> in a new module without <code>train()</code> and with <code>tf.reset_default_graph()</code> at the top, it works perfectly. How do I make it work in the same module?</p>
  
  <p>Final snippet:</p>
  
  <pre><code>if __name__ == '__main__':
      print("Start training")
      train()
      print("Finished training. Generate prediction")
      run()
  </code></pre>