{"action":"created","issue":{"url":"https://api.github.com/repos/tensorflow/models/issues/4783","repository_url":"https://api.github.com/repos/tensorflow/models","labels_url":"https://api.github.com/repos/tensorflow/models/issues/4783/labels{/name}","comments_url":"https://api.github.com/repos/tensorflow/models/issues/4783/comments","events_url":"https://api.github.com/repos/tensorflow/models/issues/4783/events","html_url":"https://github.com/tensorflow/models/issues/4783","id":341576185,"node_id":"MDU6SXNzdWUzNDE1NzYxODU=","number":4783,"title":"Quantized SSD-MobileNet Checkpoints Missing Min/Max?","user":{"login":"parvizp","id":926261,"node_id":"MDQ6VXNlcjkyNjI2MQ==","avatar_url":"https://avatars0.githubusercontent.com/u/926261?v=4","gravatar_id":"","url":"https://api.github.com/users/parvizp","html_url":"https://github.com/parvizp","followers_url":"https://api.github.com/users/parvizp/followers","following_url":"https://api.github.com/users/parvizp/following{/other_user}","gists_url":"https://api.github.com/users/parvizp/gists{/gist_id}","starred_url":"https://api.github.com/users/parvizp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/parvizp/subscriptions","organizations_url":"https://api.github.com/users/parvizp/orgs","repos_url":"https://api.github.com/users/parvizp/repos","events_url":"https://api.github.com/users/parvizp/events{/privacy}","received_events_url":"https://api.github.com/users/parvizp/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":{"login":"achowdhery","id":4723042,"node_id":"MDQ6VXNlcjQ3MjMwNDI=","avatar_url":"https://avatars3.githubusercontent.com/u/4723042?v=4","gravatar_id":"","url":"https://api.github.com/users/achowdhery","html_url":"https://github.com/achowdhery","followers_url":"https://api.github.com/users/achowdhery/followers","following_url":"https://api.github.com/users/achowdhery/following{/other_user}","gists_url":"https://api.github.com/users/achowdhery/gists{/gist_id}","starred_url":"https://api.github.com/users/achowdhery/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/achowdhery/subscriptions","organizations_url":"https://api.github.com/users/achowdhery/orgs","repos_url":"https://api.github.com/users/achowdhery/repos","events_url":"https://api.github.com/users/achowdhery/events{/privacy}","received_events_url":"https://api.github.com/users/achowdhery/received_events","type":"User","site_admin":false},"assignees":[{"login":"achowdhery","id":4723042,"node_id":"MDQ6VXNlcjQ3MjMwNDI=","avatar_url":"https://avatars3.githubusercontent.com/u/4723042?v=4","gravatar_id":"","url":"https://api.github.com/users/achowdhery","html_url":"https://github.com/achowdhery","followers_url":"https://api.github.com/users/achowdhery/followers","following_url":"https://api.github.com/users/achowdhery/following{/other_user}","gists_url":"https://api.github.com/users/achowdhery/gists{/gist_id}","starred_url":"https://api.github.com/users/achowdhery/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/achowdhery/subscriptions","organizations_url":"https://api.github.com/users/achowdhery/orgs","repos_url":"https://api.github.com/users/achowdhery/repos","events_url":"https://api.github.com/users/achowdhery/events{/privacy}","received_events_url":"https://api.github.com/users/achowdhery/received_events","type":"User","site_admin":false}],"milestone":null,"comments":4,"created_at":"2018-07-16T15:51:31Z","updated_at":"2018-07-18T18:21:42Z","closed_at":"2018-07-17T00:05:05Z","author_association":"NONE","body":"### System information\r\n- **What is the top-level directory of the model you are using**: `models/research`\r\n- **Have I written custom code (as opposed to using a stock example script provided in TensorFlow)**: No\r\n- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: Linux Ubuntu 16.04\r\n- **TensorFlow installed from (source or binary)**: Source\r\n- **TensorFlow version (use command below)**: ('v1.9.0-rc2-400-geb5af9ad02', '1.9.0-rc0')\r\n- **Bazel version (if compiling from source)**: bazel release 0.14.1\r\n- **CUDA/cuDNN version**: 9.2/7.1.4\r\n- **GPU model and memory**: NVIDIA P40/12\r\n- **Exact command to reproduce**: ```CUDA_VISIBLE_DEVICES=-1 python object_detection/export_tflite_ssd_graph.py --pipeline_config_path=object_detection/graphs/ssd_mobilenet_v1_0.75_depth_quantized_300x300_coco14_sync_2018_07_03/pipeline.config --trained_checkpoint_prefix=object_detection/graphs/ssd_mobilenet_v1_0.75_depth_quantized_300x300_coco14_sync_2018_07_03/model.ckpt --output_directory=/tmp/ssd_mobilenet_v1_0.75_depth_quantized_300x300_coco14_sync_2018_07_03/  --add_postprocessing_op=true```\r\n\r\n\r\n### Describe the problem\r\n@achowdhery  Are the \"quantized\" models in the [zoo](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md) checkpoint's supposed to contain the `FakeQuant` (Min/Max)? \r\n\r\nI tried `ssd_mobilenet_v1_0.75_depth_quantized_coco` and `ssd_mobilenet_v1_quantized_coco`. \r\n\r\nI used the command from the [tutorial](https://gist.github.com/sararob/46cc4d86624ccc2863486ee174519e10#file-run-tflite-export-sh) to export a quantized TF-Lite model. \r\n\r\n### Source code / logs\r\n```\r\nparvizp@cent-nano-0:~/Git/tensorflow.new/models/research$ CUDA_VISIBLE_DEVICES=-1 python object_detection/export_tflite_ssd_graph.py --pipeline_config_path=object_detection/graphs/ssd_mobilenet_v1_0.75_depth_quantized_300x300_coco14_sync_2018_07_03/pipeline.config --trained_checkpoint_prefix=object_detection/graphs/ssd_mobilenet_v1_0.75_depth_quantized_300x300_coco14_sync_2018_07_03/model.ckpt --output_directory=/tmp/ssd_mobilenet_v1_0.75_depth_quantized_300x300_coco14_sync_2018_07_03/  --add_postprocessing_op=true\r\n2018-07-16 10:47:11.378546: E tensorflow/stream_executor/cuda/cuda_driver.cc:397] failed call to cuInit: CUDA_ERROR_NO_DEVICE\r\n2018-07-16 10:47:11.378601: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:163] retrieving CUDA diagnostic information for host: cent-nano-0\r\n2018-07-16 10:47:11.378610: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:170] hostname: cent-nano-0\r\n2018-07-16 10:47:11.378641: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:194] libcuda reported version is: 396.26.0\r\n2018-07-16 10:47:11.378670: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:198] kernel reported version is: 396.26.0\r\n2018-07-16 10:47:11.378678: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:305] kernel version seems to match DSO: 396.26.0\r\n2018-07-16 10:47:13.255871: W tensorflow/core/framework/op_kernel.cc:1275] OP_REQUIRES failed at save_restore_v2_ops.cc:184 : Not found: Key BoxPredictor_0/BoxEncodingPredictor/act_quant/max not found in checkpoint\r\nTraceback (most recent call last):\r\n  File \"object_detection/export_tflite_ssd_graph.py\", line 137, in <module>\r\n    tf.app.run(main)\r\n  File \"/home/parvizp/.local/lib/python2.7/site-packages/tensorflow/python/platform/app.py\", line 125, in run\r\n    _sys.exit(main(argv))\r\n  File \"object_detection/export_tflite_ssd_graph.py\", line 133, in main\r\n    FLAGS.max_classes_per_detection)\r\n  File \"/home/parvizp/Git/tensorflow.new/models/research/object_detection/export_tflite_ssd_graph_lib.py\", line 261, in export_tflite_graph\r\n    initializer_nodes='')\r\n  File \"/home/parvizp/Git/tensorflow.new/models/research/object_detection/exporter.py\", line 72, in freeze_graph_with_def_protos\r\n    saver.restore(sess, input_checkpoint)\r\n  File \"/home/parvizp/.local/lib/python2.7/site-packages/tensorflow/python/training/saver.py\", line 1743, in restore\r\n    err, \"a Variable name or other graph key that is missing\")\r\ntensorflow.python.framework.errors_impl.NotFoundError: Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:\r\n\r\nKey BoxPredictor_0/BoxEncodingPredictor/act_quant/max not found in checkpoint\r\n         [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_INT64], _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]\r\n\r\nCaused by op u'save/RestoreV2', defined at:\r\n  File \"object_detection/export_tflite_ssd_graph.py\", line 137, in <module>\r\n    tf.app.run(main)\r\n  File \"/home/parvizp/.local/lib/python2.7/site-packages/tensorflow/python/platform/app.py\", line 125, in run\r\n    _sys.exit(main(argv))\r\n  File \"object_detection/export_tflite_ssd_graph.py\", line 133, in main\r\n    FLAGS.max_classes_per_detection)\r\n  File \"/home/parvizp/Git/tensorflow.new/models/research/object_detection/export_tflite_ssd_graph_lib.py\", line 261, in export_tflite_graph\r\n    initializer_nodes='')\r\n  File \"/home/parvizp/Git/tensorflow.new/models/research/object_detection/exporter.py\", line 67, in freeze_graph_with_def_protos\r\n    tf.import_graph_def(input_graph_def, name='')\r\n  File \"/home/parvizp/.local/lib/python2.7/site-packages/tensorflow/python/util/deprecation.py\", line 454, in new_func\r\n    return func(*args, **kwargs)\r\n  File \"/home/parvizp/.local/lib/python2.7/site-packages/tensorflow/python/framework/importer.py\", line 442, in import_graph_def\r\n    _ProcessNewOps(graph)\r\n  File \"/home/parvizp/.local/lib/python2.7/site-packages/tensorflow/python/framework/importer.py\", line 234, in _ProcessNewOps\r\n    for new_op in graph._add_new_tf_operations(compute_devices=False):  # pylint: disable=protected-access\r\n  File \"/home/parvizp/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py\", line 3360, in _add_new_tf_operations\r\n    for c_op in c_api_util.new_tf_operations(self)\r\n  File \"/home/parvizp/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py\", line 3251, in _create_op_from_tf_operation\r\n    ret = Operation(c_op, self)\r\n  File \"/home/parvizp/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py\", line 1716, in __init__\r\n    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access\r\n\r\nNotFoundError (see above for traceback): Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:\r\n\r\nKey BoxPredictor_0/BoxEncodingPredictor/act_quant/max not found in checkpoint\r\n         [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_INT64], _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]\r\n```"},"comment":{"url":"https://api.github.com/repos/tensorflow/models/issues/comments/406027706","html_url":"https://github.com/tensorflow/models/issues/4783#issuecomment-406027706","issue_url":"https://api.github.com/repos/tensorflow/models/issues/4783","id":406027706,"node_id":"MDEyOklzc3VlQ29tbWVudDQwNjAyNzcwNg==","user":{"login":"achowdhery","id":4723042,"node_id":"MDQ6VXNlcjQ3MjMwNDI=","avatar_url":"https://avatars3.githubusercontent.com/u/4723042?v=4","gravatar_id":"","url":"https://api.github.com/users/achowdhery","html_url":"https://github.com/achowdhery","followers_url":"https://api.github.com/users/achowdhery/followers","following_url":"https://api.github.com/users/achowdhery/following{/other_user}","gists_url":"https://api.github.com/users/achowdhery/gists{/gist_id}","starred_url":"https://api.github.com/users/achowdhery/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/achowdhery/subscriptions","organizations_url":"https://api.github.com/users/achowdhery/orgs","repos_url":"https://api.github.com/users/achowdhery/repos","events_url":"https://api.github.com/users/achowdhery/events{/privacy}","received_events_url":"https://api.github.com/users/achowdhery/received_events","type":"User","site_admin":false},"created_at":"2018-07-18T18:21:42Z","updated_at":"2018-07-18T18:21:42Z","author_association":"NONE","body":"Please give exact instructions to reproduce. Need to make sure we see same issue"}}
{"actor":{"display_login":"achowdhery"}}