title: Key <variable_name> not found in checkpoint Tensorflow
body: <p>I'm using Tensorflow v1.1 and I've been trying to figure out how to use my EMA'ed weights for inference, but no matter what I do I keep getting the error </p>
  
  <blockquote>
    <p>Not found: Key W/ExponentialMovingAverage not found in checkpoint</p>
  </blockquote>
  
  <p>even though when I loop through and print out all the <code>tf.global_variables</code> the key exists</p>
  
  <p>Here is a reproducible script heavily adapted from <a href="https://github.com/davidsandberg/facenet/blob/master/test/restore_test.py" rel="noreferrer">Facenet's</a> unit test:</p>
  
  <pre><code>import tensorflow as tf
  import numpy as np
  
  
  tf.reset_default_graph()
  
  # Create 100 phony x, y data points in NumPy, y = x * 0.1 + 0.3
  x_data = np.random.rand(100).astype(np.float32)
  y_data = x_data * 0.1 + 0.3
  
  # Try to find values for W and b that compute y_data = W * x_data + b
  # (We know that W should be 0.1 and b 0.3, but TensorFlow will
  # figure that out for us.)
  W = tf.Variable(tf.random_uniform([1], -1.0, 1.0), name='W')
  b = tf.Variable(tf.zeros([1]), name='b')
  y = W * x_data + b
  
  # Minimize the mean squared errors.
  loss = tf.reduce_mean(tf.square(y - y_data))
  optimizer = tf.train.GradientDescentOptimizer(0.5)
  opt_op = optimizer.minimize(loss)
  
  # Track the moving averages of all trainable variables.
  ema = tf.train.ExponentialMovingAverage(decay=0.9999)
  variables = tf.trainable_variables()
  print(variables)
  averages_op = ema.apply(tf.trainable_variables())
  with tf.control_dependencies([opt_op]):
      train_op = tf.group(averages_op)
  
  # Before starting, initialize the variables.  We will 'run' this first.
  init = tf.global_variables_initializer()
  
  saver = tf.train.Saver(tf.trainable_variables())
  
  # Launch the graph.
  sess = tf.Session()
  sess.run(init)
  
  # Fit the line.
  for _ in range(201):
      sess.run(train_op)
  
  w_reference = sess.run('W/ExponentialMovingAverage:0')
  b_reference = sess.run('b/ExponentialMovingAverage:0')
  
  saver.save(sess, os.path.join("model_ex1"))
  
  tf.reset_default_graph()
  
  tf.train.import_meta_graph("model_ex1.meta")
  sess = tf.Session()
  
  print('------------------------------------------------------')
  for var in tf.global_variables():
      print('all variables: ' + var.op.name)
  for var in tf.trainable_variables():
      print('normal variable: ' + var.op.name)
  for var in tf.moving_average_variables():
      print('ema variable: ' + var.op.name)
  print('------------------------------------------------------')
  
  mode = 1
  restore_vars = {}
  if mode == 0:
      ema = tf.train.ExponentialMovingAverage(1.0)
      for var in tf.trainable_variables():
          print('%s: %s' % (ema.average_name(var), var.op.name))
          restore_vars[ema.average_name(var)] = var
  elif mode == 1:
      for var in tf.trainable_variables():
          ema_name = var.op.name + '/ExponentialMovingAverage'
          print('%s: %s' % (ema_name, var.op.name))
          restore_vars[ema_name] = var
  
  saver = tf.train.Saver(restore_vars, name='ema_restore')
  
  saver.restore(sess, os.path.join("model_ex1")) # error happens here!
  
  w_restored = sess.run('W:0')
  b_restored = sess.run('b:0')
  
  print(w_reference)
  print(w_restored)
  print(b_reference)
  print(b_restored)
  </code></pre>