title: How to properly serve an object detection model from Tensorflow Object Detection API?
body: <p>I am using Tensorflow Object Detection API(github.com/tensorflow/models/tree/master/object_detection) with one object detection task. Right now I am having problem on serving the detection model I trained with Tensorflow Serving(tensorflow.github.io/serving/).</p>
  
  <p><strong>1.</strong> The first issue I am encountering is about exporting the model to servable files.
  The object detection api kindly included the export script so that I am able to convert ckpt files to pb files with variables. However, the output files will not have any content in 'variables' folder. I though this was a bug and reported it on Github, but it seems they interned to convert variables to constants so that there will be no variables. The detail can be found <a href="https://github.com/tensorflow/models/issues/2045" rel="noreferrer">HERE</a>.</p>
  
  <p>The flags I was using when exporting the saved model is as follows:</p>
  
  <pre><code>    CUDA_VISIBLE_DEVICES=0 python export_inference_graph.py \
          --input_type image_tensor \
              --pipeline_config_path configs/rfcn_resnet50_car_Jul_20.config \
                  --checkpoint_path resnet_ckpt/model.ckpt-17586 \
                      --inference_graph_path serving_model/1 \
                        --export_as_saved_model True
  </code></pre>
  
  <p>It runs perfectly fine in python when I switch --export_as_saved_model to False.</p>
  
  <p>But still, I am having issue with serving the model. </p>
  
  <p>When I was trying to run:</p>
  
  <pre><code>~/serving$ bazel-bin/tensorflow_serving/model_servers/tensorflow_model_server --port=9000 --model_name=gan --model_base_path=&lt;my_model_path&gt;
  </code></pre>
  
  <p>I got:</p>
  
  <pre><code>2017-07-27 16:11:53.222439: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:155] Restoring SavedModel bundle.
  2017-07-27 16:11:53.222497: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:165] The specified SavedModel has no variables; no checkpoints were restored.
  2017-07-27 16:11:53.222502: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:190] Running LegacyInitOp on SavedModel bundle.
  2017-07-27 16:11:53.229463: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:284] Loading SavedModel: success. Took 281805 microseconds.
  2017-07-27 16:11:53.229508: I tensorflow_serving/core/loader_harness.cc:86] Successfully loaded servable version {name: gan version: 1}
  2017-07-27 16:11:53.244716: I tensorflow_serving/model_servers/main.cc:290] Running ModelServer at 0.0.0.0:9000 ...
  </code></pre>
  
  <p>I think the model was not properly loaded since it shows "The specified SavedModel has no variables; no checkpoints were restored."</p>
  
  <p>But since we have converted all variables into constants, it seems reasonable. I am not sure here.</p>
  
  <p><strong>2</strong>. I was not able to use client to call server and do detection on a sample image.</p>
  
  <p>The client scrip is listed below:</p>
  
  <pre><code>from __future__ import print_function
  from __future__ import absolute_import
  
  # Communication to TensorFlow server via gRPC
  from grpc.beta import implementations
  import tensorflow as tf
  import numpy as np
  from PIL import Image
  # TensorFlow serving stuff to send messages
  from tensorflow_serving.apis import predict_pb2
  from tensorflow_serving.apis import prediction_service_pb2
  
  
  # Command line arguments
  tf.app.flags.DEFINE_string('server', 'localhost:9000',
                         'PredictionService host:port')
  tf.app.flags.DEFINE_string('image', '', 'path to image in JPEG format')
  FLAGS = tf.app.flags.FLAGS
  
  
  def load_image_into_numpy_array(image):
      (im_width, im_height) = image.size
      return np.array(image.getdata()).reshape(
      (im_height, im_width, 3)).astype(np.uint8)
  
  def main(_):
      host, port = FLAGS.server.split(':')
      channel = implementations.insecure_channel(host, int(port))
      stub = prediction_service_pb2.beta_create_PredictionService_stub(channel)
  
      # Send request
      request = predict_pb2.PredictRequest()
      image = Image.open(FLAGS.image)
      image_np = load_image_into_numpy_array(image)
      image_np_expanded = np.expand_dims(image_np, axis=0)
      # Call GAN model to make prediction on the image
      request.model_spec.name = 'gan'
      request.model_spec.signature_name = 'predict_images'
      request.inputs['inputs'].CopyFrom(
      tf.contrib.util.make_tensor_proto(image_np_expanded))
  
      result = stub.Predict(request, 60.0)  # 60 secs timeout
      print(result)
  
  
  if __name__ == '__main__':
      tf.app.run()
  </code></pre>
  
  <p>To match <code>request.model_spec.signature_name = 'predict_images'</code>, I modified the exporter.py script in object detection api (github.com/tensorflow/models/blob/master/object_detection/exporter.py) started at line 289 from:</p>
  
  <pre><code>          signature_def_map={
            signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY:
                detection_signature,
        },
  </code></pre>
  
  <p>To:</p>
  
  <pre><code>          signature_def_map={
            'predict_images': detection_signature,
            signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY:
                detection_signature,
        },
  </code></pre>
  
  <p>Since I have no idea how to call a default signature key.</p>
  
  <p>When I run the following command:</p>
  
  <pre><code>bazel-bin/tensorflow_serving/example/client --server=localhost:9000 --image=&lt;my_image_file&gt;
  </code></pre>
  
  <p>I got following error message:</p>
  
  <pre><code>    Traceback (most recent call last):
    File "/home/xinyao/serving/bazel-bin/tensorflow_serving/example/client.runfiles/tf_serving/tensorflow_serving/example/client.py", line 54, in &lt;module&gt;
      tf.app.run()
    File "/home/xinyao/serving/bazel-bin/tensorflow_serving/example/client.runfiles/org_tensorflow/tensorflow/python/platform/app.py", line 48, in run
      _sys.exit(main(_sys.argv[:1] + flags_passthrough))
    File "/home/xinyao/serving/bazel-bin/tensorflow_serving/example/client.runfiles/tf_serving/tensorflow_serving/example/client.py", line 49, in main
      result = stub.Predict(request, 60.0)  # 60 secs timeout
    File "/usr/local/lib/python2.7/dist-packages/grpc/beta/_client_adaptations.py", line 324, in __call__
      self._request_serializer, self._response_deserializer)
    File "/usr/local/lib/python2.7/dist-packages/grpc/beta/_client_adaptations.py", line 210, in _blocking_unary_unary
      raise _abortion_error(rpc_error_call)
  grpc.framework.interfaces.face.face.AbortionError: AbortionError(code=StatusCode.NOT_FOUND, details="FeedInputs: unable to find feed output ToFloat:0")
  </code></pre>
  
  <p>Not quite sure what's going on here.</p>
  
  <p>Initially I though maybe my client script is not correct, after I found the AbortionError is from github.com/tensorflow/tensorflow/blob/f488419cd6d9256b25ba25cbe736097dfeee79f9/tensorflow/core/graph/subgraph.cc. Seems I got this error when building the graph. So it might be caused by the first issue I have.</p>
  
  <p>I am new to this stuff, so I am really confused. I think I might be wrong at start. Is there any way that I could properly export and serve the detection model? Any suggestions will be of great help!</p>