{"action":"created","issue":{"url":"https://api.github.com/repos/tensorflow/serving/issues/777","repository_url":"https://api.github.com/repos/tensorflow/serving","labels_url":"https://api.github.com/repos/tensorflow/serving/issues/777/labels{/name}","comments_url":"https://api.github.com/repos/tensorflow/serving/issues/777/comments","events_url":"https://api.github.com/repos/tensorflow/serving/issues/777/events","html_url":"https://github.com/tensorflow/serving/issues/777","id":299566254,"node_id":"MDU6SXNzdWUyOTk1NjYyNTQ=","number":777,"title":"Seq2Seq Model Serialization and Deserialization Problem","user":{"login":"bugra","id":372676,"node_id":"MDQ6VXNlcjM3MjY3Ng==","avatar_url":"https://avatars2.githubusercontent.com/u/372676?v=4","gravatar_id":"","url":"https://api.github.com/users/bugra","html_url":"https://github.com/bugra","followers_url":"https://api.github.com/users/bugra/followers","following_url":"https://api.github.com/users/bugra/following{/other_user}","gists_url":"https://api.github.com/users/bugra/gists{/gist_id}","starred_url":"https://api.github.com/users/bugra/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bugra/subscriptions","organizations_url":"https://api.github.com/users/bugra/orgs","repos_url":"https://api.github.com/users/bugra/repos","events_url":"https://api.github.com/users/bugra/events{/privacy}","received_events_url":"https://api.github.com/users/bugra/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-02-23T01:12:09Z","updated_at":"2018-08-03T20:09:01Z","closed_at":"2018-08-03T20:09:01Z","author_association":"NONE","body":"\r\n\r\nI am already using inception like model for image prediction in tensorflow serving in production and would like to use a tensorflow serving for a `seq2seq` model. However, I have a problem with serializing/deserializing a `tf_servable` in the tensorflow serving layer.\r\n\r\nI am using `tf.data.TFRecordDataset` interface to train a Seq2Seq Model where the `valid['sample']` is a TFRecordDataSet and `valid['model']` is my seq2seq model:\r\n\r\n```python\r\noutput = sess.run([valid['model'].beam_output] + list(valid['sample'].values()))\r\noutput, data = output[0], output[1:]\r\ndata = {k:v for k,v in zip(valid['sample'].keys(), data)}\r\noutput = output.predicted_ids[0].T\r\n```\r\nwhere data looks like this:\r\n```\r\n{'action': array([[1., 1., 1., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\r\n         0., 1., 1., 1., 1., 1., 1., 0., 0., 0.]], dtype=float32),\r\n 'add_to_cart': array([[1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,\r\n         1., 1., 1., 1., 1., 1., 1., 0., 0., 0.]], dtype=float32),\r\n 'copy_labels': array([[1, 0, 0, 1, 1]]),\r\n 'eos_inds': array([[ 5, 11, 16, 22, 25]]),\r\n 'query_context': array([[   85,   115,    69,   370, 23373,     1,   515,   366,    75,\r\n            19,   233,     1,   453,    75,    19,   233,     1, 18772,\r\n            33, 21360,  2028,  1621,     1,  1340,   692,     1]]),\r\n 'query_future': array([[1340,  995,  678,  692,    1]]),\r\n 'query_inds': array([[0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3,\r\n         3, 4, 4, 4]]),\r\n 'raw_context': array([[b'outdoor products red backpack 30l | mrs meyers hand soap refill | lemon hand soap refill | scala womens cloche wool hats | turmeric capsules']],\r\n       dtype=object),\r\n 'raw_future': array([[b'turmeric 1000 mg capsules']], dtype=object),\r\n 'time_features': array([[14.680066 , 14.680066 , 14.680066 , 14.680066 , 14.680066 ,\r\n         14.680066 , 13.885922 , 13.885922 , 13.885922 , 13.885922 ,\r\n         13.885922 , 13.885922 , 13.885863 , 13.885863 , 13.885863 ,\r\n         13.885863 , 13.885863 , 12.056493 , 12.056493 , 12.056493 ,\r\n         12.056493 , 12.056493 , 12.056493 ,  4.2195077,  4.2195077,\r\n          4.2195077]], dtype=float32),\r\n 'time_weight': array([1.], dtype=float32)}\r\n```\r\nand output is a numpy array:\r\n```\r\narray([[1340,    1,    1,    1,    1],\r\n       [6840,  692,    1,    1,    1],\r\n       [  20, 1340,  692,    1,    1],\r\n       [4113,   16, 5708,    1,    1],\r\n       [1340,  692, 5708,    1,    1],\r\n       [1340,  692, 7655,    1,    1],\r\n       [1340,  692,   20,    1,    1],\r\n       [1340,  692,  995,  678,    1],\r\n       [ 303, 1311, 1340,  692,    1],\r\n       [ 303,  183, 1340,  692,    1]], dtype=int32)\r\n```\r\n\r\nSo far so good. I can also save and load model into memory to reproduce this very easily. I do not have any problem with this.\r\n\r\nHowever, I am not sure what I am doing for serializing this model into `tf_servable` is correct:\r\n\r\n```python\r\nexport_path = os.path.join(\r\n  tf.compat.as_bytes(export_base_path),\r\n  tf.compat.as_bytes(str(version_number))\r\n)\r\n\r\nsess = tf.Session()\r\ntf.reset_default_graph()\r\nsaver = tf.train.import_meta_graph(model_meta_path)\r\n\r\n\r\nbuilder = tf.saved_model.builder.SavedModelBuilder(export_path)\r\n\r\nfeature_configs = {\r\n    'x': tf.FixedLenFeature(shape=[], dtype=tf.string),\r\n    'y': tf.FixedLenFeature(shape=[], dtype=tf.string),\r\n}\r\n\r\nserialized_example = tf.placeholder(tf.string, name=\"tf_example\")\r\ntf_example = tf.parse_example(serialized_example, feature_configs)\r\nx = tf.identity(tf_example['x'], name='x')\r\ny = tf.identity(tf_example['y'], name='y')\r\npredict_signature_def_map = tf.saved_model.signature_def_utils.predict_signature_def(\r\n    inputs={\r\n        tf.saved_model.signature_constants.PREDICT_INPUTS: x\r\n    },\r\n    outputs={\r\n        tf.saved_model.signature_constants.PREDICT_OUTPUTS: y\r\n    }\r\n)\r\n\r\nlegacy_init_op = tf.group(tf.tables_initializer(), \r\n                          name=\"legacy_init_op\")\r\n\r\nbuilder.add_meta_graph_and_variables(\r\n    sess=sess,\r\n    tags=[tf.saved_model.tag_constants.SERVING],\r\n    signature_def_map={\r\n        \"predict_signature_map\": predict_signature_def_map\r\n    },\r\n    legacy_init_op=legacy_init_op,\r\n    \r\n)\r\n\r\nbuilder.save()\r\n```\r\n\r\nThis does not specify the dataset that I feed, therefore in the deserialization within the `inputs` portion, it should be something different. I am only specifying the `inputs` broadly, but what I want to specify is actually the dictionary structure.\r\n\r\n```\r\ndict_keys(['action', 'query_inds', 'add_to_cart', 'eos_inds', 'query_future', 'time_features', 'raw_future', 'raw_context', 'time_weight', 'copy_labels', 'query_context'])\r\n```\r\nMy dictionary keys are like this and I have correct placeholder types for this dict.\r\n\r\nWhen I am trying to deserialize this model and pass a dictionary:\r\n\r\n```python\r\ndef get_prediction_response(stub,\r\n                            data,\r\n                            model_name=MODEL_NAME,\r\n                            signature_name=SIGNATURE_NAME,\r\n                            timeout_in_seconds=TIMEOUT_IN_SECONDS):\r\n  request = predict_pb2.PredictRequest()\r\n  request.model_spec.name = model_name\r\n  request.model_spec.signature_name = signature_name\r\n  request.inputs['inputs'].CopyFrom(tf.contrib.util.make_tensor_proto(data, shape=[1]))\r\n\r\n  return stub.Predict(request, timeout_in_seconds) \r\n```\r\n\r\nI am trying to pass either a string or a dictionary in the data portion, but I am getting `input does not match size`. Can someone help me to define the classification signature so that serializing and deserialzing would match?\r\n\r\nIdeally, I would like to feed a dictionary to `tf_servable` model and would like to get the prediction."},"comment":{"url":"https://api.github.com/repos/tensorflow/serving/issues/comments/410362978","html_url":"https://github.com/tensorflow/serving/issues/777#issuecomment-410362978","issue_url":"https://api.github.com/repos/tensorflow/serving/issues/777","id":410362978,"node_id":"MDEyOklzc3VlQ29tbWVudDQxMDM2Mjk3OA==","user":{"login":"gautamvasudevan","id":12356580,"node_id":"MDQ6VXNlcjEyMzU2NTgw","avatar_url":"https://avatars3.githubusercontent.com/u/12356580?v=4","gravatar_id":"","url":"https://api.github.com/users/gautamvasudevan","html_url":"https://github.com/gautamvasudevan","followers_url":"https://api.github.com/users/gautamvasudevan/followers","following_url":"https://api.github.com/users/gautamvasudevan/following{/other_user}","gists_url":"https://api.github.com/users/gautamvasudevan/gists{/gist_id}","starred_url":"https://api.github.com/users/gautamvasudevan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gautamvasudevan/subscriptions","organizations_url":"https://api.github.com/users/gautamvasudevan/orgs","repos_url":"https://api.github.com/users/gautamvasudevan/repos","events_url":"https://api.github.com/users/gautamvasudevan/events{/privacy}","received_events_url":"https://api.github.com/users/gautamvasudevan/received_events","type":"User","site_admin":false},"created_at":"2018-08-03T20:09:01Z","updated_at":"2018-08-03T20:09:01Z","author_association":"CONTRIBUTOR","body":"Closing due to staleness. If this is still an issue, please file a new updated issue with current steps to reproduce the bug. If this is a question, please ask it on: \r\n\r\nhttps://stackoverflow.com/questions/tagged/tensorflow-serving\r\n\r\nThanks!"}}
{"actor":{"display_login":"gautamvasudevan"}}