title: How to restore my loss from a saved meta graph?
body: <p>I have built a simple tensorflow model that is working fine.
  While training I save the meta_graph and also some parameters at different steps.</p>
  
  <p>After that (in a new script) I want to restore the saved meta_graph and restore variables and operations.</p>
  
  <p>Everything works fine, but only the </p>
  
  <pre><code>with tf.name_scope('MSE'):
      error = tf.losses.mean_squared_error(Y, yhat, scope="error")
  </code></pre>
  
  <p>is not going to be restored. With the following line</p>
  
  <pre><code>mse_error = graph.get_tensor_by_name("MSE/error:0")
  </code></pre>
  
  <blockquote>
    <p>"The name 'MSE/error:0' refers to a Tensor which does not exist. The
    operation, 'MSE/error', does not exist in the graph."</p>
  </blockquote>
  
  <p>there appears this error message.</p>
  
  <p>As I do exactly the same procedure for other variables and ops that are restored without any error, I don't know how to deal with that. Only difference is that there is only a scope attribute and not a name attribute in the tf.losses.mean_squared_error function.</p>
  
  <p>So how do I restore the loss operation with the scope?</p>
  
  <p>Here the code how I save and load the model.</p>
  
  <p>Saving:</p>
  
  <pre><code># define network ...
  saver = tf.train.Saver(max_to_keep=10)
  
  sess = tf.Session()
  sess.run(tf.global_variables_initializer())
  
  for i in range(NUM_EPOCHS):
      # do training ..., save model all 1000 optimization steps
      if (i + 1) % 1000 == 0:
          saver.save(sess, "L:/model/mlp_model", global_step=(i+1))
  </code></pre>
  
  <p>Restore:</p>
  
  <pre><code># start a session
  sess=tf.Session()
  # load meta graph
  saver = tf.train.import_meta_graph('L:\\model\\mlp_model-1000.meta')
  # restore weights
  saver.restore(sess, tf.train.latest_checkpoint('L:\\model\\'))
  
  # access network nodes
  graph = tf.get_default_graph()
  X = graph.get_tensor_by_name("Input/X:0")
  Y = graph.get_tensor_by_name("Input/Y:0")
  
  # restore output-generating operation used for prediction
  yhat_op = graph.get_tensor_by_name("OutputLayer/yhat:0")
  mse_error = graph.get_tensor_by_name("MSE/error:0") # this one doesn't work
  </code></pre>