title: Restoring a model trained with tf.estimator and feeding input through feed_dict
body: <p>I trained a resnet with tf.estimator, the model was saved during the training process. The saved files consist of <code>.data</code>, <code>.index</code> and <code>.meta</code>. I'd like to load this model back and get predictions for new images. The data was fed to the model during training using <code>tf.data.Dataset</code>.  I have closely followed the resnet implementation given <a href="https://github.com/deepaksuresh/models/blob/master/official/resnet/resnet.py" rel="nofollow noreferrer">here</a>.</p>
  
  <p>I would like to restore the model and feed inputs to the nodes using a feed_dict. </p>
  
  <p><strong>First attempt</strong> </p>
  
  <pre><code>  #rebuild input pipeline
    images, labels = input_fn(data_dir, batch_size=32, num_epochs=1)
  
    #rebuild graph
    prediction= imagenet_model_fn(images,labels,{'batch_size':32,'data_format':'channels_first','resnet_size':18},mode = tf.estimator.ModeKeys.EVAL).predictions 
  
    saver  = tf.train.Saver()
    with tf.Session() as sess:
      ckpt = tf.train.get_checkpoint_state(r'./model')
      saver.restore(sess, ckpt.model_checkpoint_path)
      while True:
      try:
          pred,im= sess.run([prediction,images])
          print(pred)
      except tf.errors.OutOfRangeError:
        break
  </code></pre>
  
  <p>I fed a dataset which was evaluated on the same model using <code>classifier.evaluate</code>, but the above method gives wrong predictions. The model gives same class and probability, 1.0, for all images.</p>
  
  <p><strong>Second attempt</strong></p>
  
  <pre><code>saver = tf.train.import_meta_graph(r'.\resnet\model\model-3220.meta')
  sess = tf.Session()
  saver.restore(sess,tf.train.latest_checkpoint(r'.\resnet\model'))
  graph = tf.get_default_graph()
  inputImage = graph.get_tensor_by_name('image:0')
  logits= graph.get_tensor_by_name('logits:0')
  
  #Get prediction
  print(sess.run(logits,feed_dict={inputImage:newimage}))
  </code></pre>
  
  <p>This also gives wrong predictions compared to <code>classifier.evaluate</code>. I can even run <code>sess.run(logits)</code> without a <code>feed_dict</code>!</p>
  
  <p><strong>Third attempt</strong></p>
  
  <pre><code>def serving_input_fn():
    receiver_tensor = {'feature': tf.placeholder(shape=[None, 384, 256, 3], dtype=tf.float32)}
    features = {'feature': receiver_tensor['images']}
  return tf.estimator.export.ServingInputReceiver(features, receiver_tensor)
  </code></pre>
  
  <p>It fails with </p>
  
  <pre><code>Traceback (most recent call last):
    File "imagenet_main.py", line 213, in &lt;module&gt;
      tf.app.run(argv=[sys.argv[0]] + unparsed)
    File "C:\Users\Photogauge\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\platform\app.py", line 124, in run
      _sys.exit(main(argv))
    File "imagenet_main.py", line 204, in main
      resnet.resnet_main(FLAGS, imagenet_model_fn, input_fn)
    File "C:\Users\Photogauge\Desktop\iprings_images\models-master\models-master\official\resnet\resnet.py", line 527, in resnet_main
      classifier.export_savedmodel(export_dir_base=r"C:\Users\Photogauge\Desktop\iprings_images\models-master\models-master\official\resnet\export", serving_input_receiver_fn=serving_input_fn)
    File "C:\Users\Photogauge\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\estimator\estimator.py", line 528, in export_savedmodel
      config=self.config)
    File "C:\Users\Photogauge\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\estimator\estimator.py", line 725, in _call_model_fn
      model_fn_results = self._model_fn(features=features, **kwargs)
    File "imagenet_main.py", line 200, in imagenet_model_fn
      loss_filter_fn=None)
    File "C:\Users\Photogauge\Desktop\iprings_images\models-master\models-master\official\resnet\resnet.py", line 433, in resnet_model_fn
      tf.argmax(labels, axis=1), predictions['classes'])
    File "C:\Users\Photogauge\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\util\deprecation.py", line 316, in new_func
      return func(*args, **kwargs)
    File "C:\Users\Photogauge\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\ops\math_ops.py", line 208, in argmax
      return gen_math_ops.arg_max(input, axis, name=name, output_type=output_type)
    File "C:\Users\Photogauge\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\ops\gen_math_ops.py", line 508, in arg_max
      name=name)
    File "C:\Users\Photogauge\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 528, in _apply_op_helper
      (input_name, err))
  ValueError: Tried to convert 'input' to a tensor and failed. Error: None values not supported.
  </code></pre>
  
  <p>The code I used for training and building the model is as below:</p>
  
  <p>Specification for parsing the dataset:</p>
  
  <pre><code>def parse_record(raw_record, is_training):
    keys_to_features = {
        'image/encoded':
            tf.FixedLenFeature((), tf.string, default_value=''),
        'image/class/label':
            tf.FixedLenFeature([], dtype=tf.int64, default_value=-1),
    }
    parsed = tf.parse_single_example(raw_record, keys_to_features)
    image = tf.image.decode_image(
        tf.reshape(parsed['image/encoded'], shape=[]),3)
    image = tf.image.convert_image_dtype(image, dtype=tf.float32)
    label = tf.cast(
        tf.reshape(parsed['image/class/label'], shape=[]),
        dtype=tf.int32)
    return image, tf.one_hot(label,2)
  </code></pre>
  
  <p>The following function parses the data and creates batches for training</p>
  
  <pre><code>def input_fn(is_training, data_dir, batch_size, num_epochs=1):
    dataset = tf.data.Dataset.from_tensor_slices(
        filenames(is_training, data_dir))
    if is_training:
       dataset = dataset.shuffle(buffer_size=_FILE_SHUFFLE_BUFFER)
    dataset = dataset.flat_map(tf.data.TFRecordDataset)
    dataset = dataset.map(lambda value: parse_record(value, is_training),
                          num_parallel_calls=5)
    dataset = dataset.prefetch(batch_size)
    if is_training:
        dataset = dataset.shuffle(buffer_size=_SHUFFLE_BUFFER)
    dataset = dataset.repeat(num_epochs)
    dataset = dataset.batch(batch_size)
  
    iterator = dataset.make_one_shot_iterator()
    images, labels = iterator.get_next()
    return images, labels
  </code></pre>
  
  <p>A classifier is created as below for training on train set and evaluation on validation set</p>
  
  <pre><code>classifier = tf.estimator.Estimator(
        model_fn=model_function, model_dir=flags.model_dir, config=run_config,
        params={
            'resnet_size': flags.resnet_size,
            'data_format': flags.data_format,
            'batch_size': flags.batch_size,
        })
  
      #Training cycle
       classifier.train(
           input_fn=lambda: input_function(
               training_phase=True, flags.data_dir, flags.batch_size, flags.epochs_per_eval),
           hooks=[logging_hook])
      # Evaluate the model 
      eval_results = classifier.evaluate(input_fn=lambda: input_function(
          training_phase=False, flags.data_dir, flags.batch_size))
  </code></pre>
  
  <p>This is how I tried to load and get predictions from the model.</p>
  
  <p>What is the right way to restore a saved model and perform inference on it. I want to feed images directly without using <code>tf.data.Dataset</code>. </p>
  
  <p><strong>Update</strong></p>
  
  <ol>
  <li><p>The value of <code>ckpt</code> is after running <code>ckpt = tf.train.get_checkpoint_state(r'./model')</code> is </p>
  
  <p>model_checkpoint_path: "./model\model.ckpt-5980"
  all_model_checkpoint_paths: "./model\model.ckpt-5060"
  all_model_checkpoint_paths: "./model\model.ckpt-5061"
  all_model_checkpoint_paths: "./model\model.ckpt-5520"
  all_model_checkpoint_paths: "./model\model.ckpt-5521"
  all_model_checkpoint_paths: "./model\model.ckpt-5980"</p></li>
  <li><p>The output is same when I try `saver.restore(sess, tf.train.latest_checkpoint(r'.\resnet\model'))</p></li>
  <li><p>Passing in full path to <code>saver.restore</code> gives the same output
  In all cases the same model, <code>model.ckpt-5980</code> was restored</p></li>
  </ol>