{"action":"created","issue":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/18774","repository_url":"https://api.github.com/repos/tensorflow/tensorflow","labels_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/18774/labels{/name}","comments_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/18774/comments","events_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/18774/events","html_url":"https://github.com/tensorflow/tensorflow/issues/18774","id":316585532,"node_id":"MDU6SXNzdWUzMTY1ODU1MzI=","number":18774,"title":"mobilenet_v1_eval.py has a big bug?","user":{"login":"GarryLau","id":14019923,"node_id":"MDQ6VXNlcjE0MDE5OTIz","avatar_url":"https://avatars1.githubusercontent.com/u/14019923?v=4","gravatar_id":"","url":"https://api.github.com/users/GarryLau","html_url":"https://github.com/GarryLau","followers_url":"https://api.github.com/users/GarryLau/followers","following_url":"https://api.github.com/users/GarryLau/following{/other_user}","gists_url":"https://api.github.com/users/GarryLau/gists{/gist_id}","starred_url":"https://api.github.com/users/GarryLau/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/GarryLau/subscriptions","organizations_url":"https://api.github.com/users/GarryLau/orgs","repos_url":"https://api.github.com/users/GarryLau/repos","events_url":"https://api.github.com/users/GarryLau/events{/privacy}","received_events_url":"https://api.github.com/users/GarryLau/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"tatatodd","id":5453737,"node_id":"MDQ6VXNlcjU0NTM3Mzc=","avatar_url":"https://avatars3.githubusercontent.com/u/5453737?v=4","gravatar_id":"","url":"https://api.github.com/users/tatatodd","html_url":"https://github.com/tatatodd","followers_url":"https://api.github.com/users/tatatodd/followers","following_url":"https://api.github.com/users/tatatodd/following{/other_user}","gists_url":"https://api.github.com/users/tatatodd/gists{/gist_id}","starred_url":"https://api.github.com/users/tatatodd/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tatatodd/subscriptions","organizations_url":"https://api.github.com/users/tatatodd/orgs","repos_url":"https://api.github.com/users/tatatodd/repos","events_url":"https://api.github.com/users/tatatodd/events{/privacy}","received_events_url":"https://api.github.com/users/tatatodd/received_events","type":"User","site_admin":false},"assignees":[{"login":"tatatodd","id":5453737,"node_id":"MDQ6VXNlcjU0NTM3Mzc=","avatar_url":"https://avatars3.githubusercontent.com/u/5453737?v=4","gravatar_id":"","url":"https://api.github.com/users/tatatodd","html_url":"https://github.com/tatatodd","followers_url":"https://api.github.com/users/tatatodd/followers","following_url":"https://api.github.com/users/tatatodd/following{/other_user}","gists_url":"https://api.github.com/users/tatatodd/gists{/gist_id}","starred_url":"https://api.github.com/users/tatatodd/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tatatodd/subscriptions","organizations_url":"https://api.github.com/users/tatatodd/orgs","repos_url":"https://api.github.com/users/tatatodd/repos","events_url":"https://api.github.com/users/tatatodd/events{/privacy}","received_events_url":"https://api.github.com/users/tatatodd/received_events","type":"User","site_admin":false}],"milestone":null,"comments":3,"created_at":"2018-04-22T14:53:19Z","updated_at":"2018-08-24T21:50:48Z","closed_at":null,"author_association":"NONE","body":"### System information\r\n- **Have I written custom code**:Yes\r\n- **OS Platform and Distribution**:Linux Ubuntu 16.04\r\n- **TensorFlow installed from**:binary(Anaconda)\r\n- **TensorFlow version**:1.7\r\n- **Python version**: Python 3.6.2\r\n- **Bazel version**:\r\nBuild label: 0.11.1\r\nBuild target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar\r\nBuild time: Tue May 14 07:48:23 +50148 (1520362424903)\r\nBuild timestamp: 1520362424903\r\nBuild timestamp as int: 1520362424903\r\n- **CUDA/cuDNN version**:CUDA 9.0, cuDNN v7.0\r\n- **GPU model and memory**:GTX 1080Ti, 11GB\r\n- **Exact command to reproduce**:Just run mobilenet_v1_eval.py to evaluate the trained model, you will find the bug. If **is_training=True you can get the right result**.[The is_trainging are in mobilenet_v1.mobilenet_v1_arg_scope(is_training=True) and mobilenet_v1.mobilenet_v1(inputs, is_training=True) ]. **Or remove \"slim.batch_norm\" in \"with slim.arg_scope([slim.batch_norm, slim.dropout],\" can also get the right result.** \"with slim.arg_scope([slim.batch_norm, slim.dropout],\"  is in L362 of https://github.com/tensorflow/models/blob/master/research/slim/nets/mobilenet_v1.py\r\n\r\n### Describe the problem\r\nTotally, I have two problems.\r\n**Firstly**,\r\nIn the mobilenet_v1_eval.py, yeah, it's eval not train. When is_training=False, you will not get a right result. When is_training=True, you can get a right answer. Obviously, it's wrong! I guess it's a big bug associate with batch norm, but I cannot figure it out. Someone knows that? \r\nAttentation, the is_training is not the one in  flower_input(is_training=False), they are in mobilenet_v1.mobilenet_v1_arg_scope(is_training=True) and mobilenet_v1.mobilenet_v1(inputs, is_training=True). And **flower_input() was a custom code referencing imagenet_input()** in https://github.com/tensorflow/models/blob/master/research/slim/nets/mobilenet_v1_eval.py\r\n```python\r\ndef build_model():\r\n  \"\"\"Build the mobilenet_v1 model for evaluation.\r\n\r\n  Returns:\r\n    g: graph with rewrites after insertion of quantization ops and batch norm\r\n    folding.\r\n    eval_ops: eval ops for inference.\r\n    variables_to_restore: List of variables to restore from checkpoint.\r\n  \"\"\"\r\n  g = tf.Graph()\r\n  with g.as_default():\r\n    inputs, labels = flower_input(is_training=False)\r\n    scope = mobilenet_v1.mobilenet_v1_arg_scope(\r\n        is_training=True, weight_decay=0.0)\r\n    with slim.arg_scope(scope):\r\n      logits, _ = mobilenet_v1.mobilenet_v1(\r\n          inputs,\r\n          is_training=True,\r\n          depth_multiplier=FLAGS.depth_multiplier,\r\n          num_classes=FLAGS.num_classes)\r\n\r\n    if FLAGS.quantize:\r\n      tf.contrib.quantize.create_eval_graph()\r\n\r\n    eval_ops = metrics(logits, labels)\r\n\r\n  return g, eval_ops\r\n``` \r\n**Secondly**,\r\nI use export_eval_pbtxt() in the following to get the \"mobilenet_v1_eval.pbtxt\".\r\n```python\r\ndef export_eval_pbtxt():\r\n  \"\"\"Export eval.pbtxt.\"\"\"\r\n  g = tf.Graph()\r\n  with g.as_default():\r\n    inputs = tf.placeholder(dtype=tf.float32,shape=[None,224,224,3])\r\n    scope = mobilenet_v1.mobilenet_v1_arg_scope(\r\n        is_training=False, weight_decay=0.0)\r\n    with slim.arg_scope(scope):\r\n      logits, _ = mobilenet_v1.mobilenet_v1(\r\n          inputs,\r\n          is_training=False,\r\n          depth_multiplier=FLAGS.depth_multiplier,\r\n          num_classes=FLAGS.num_classes)\r\n    eval_graph_file = '/home/lg/projects/mobilenet_v1_eval.pbtxt'\r\n    with tf.Session() as sess:\r\n          with open(eval_graph_file, 'w') as f:\r\n            f.write(str(g.as_graph_def()))\r\n\r\n```\r\nThen, frozen the graph:\r\n\r\n```python\r\nbazel-bin/tensorflow/python/tools/freeze_graph  \\\r\n  --input_graph=/home/lg/projects/mobilenet_v1_eval.pbtxt \\\r\n  --input_checkpoint=/home/lg/projects/checkpoint/model.ckpt-20000 \\\r\n  --input_binary=false \\\r\n  --output_graph=/home/lg/projects/frozen_mobilenet_v1_224.pb  \\\r\n  --output_node_names=MobilenetV1/Predictions/Reshape_1  \\\r\n  --checkpoint_version=2\r\n```\r\n\r\nThen I use the frozen .pb file to classify an image, it won't get a right result, no matter you use is_training is True or False above to get the \"mobilenet_v1_eval.pbtxt\".\r\nAnd the classify file is:\r\n```python\r\nfrom __future__ import absolute_import\r\nfrom __future__ import division\r\nfrom __future__ import print_function\r\n\r\nimport argparse\r\nimport os.path\r\nimport re\r\nimport sys\r\nimport tarfile\r\n\r\nimport numpy as np\r\nfrom six.moves import urllib\r\nimport tensorflow as tf\r\nFLAGS = None\r\n\r\n\r\ndef create_graph():\r\n  \"\"\"Creates a graph from saved GraphDef file and returns a saver.\"\"\"\r\n  # Creates graph from saved graph_def.pb.\r\n  with tf.gfile.FastGFile(os.path.join(FLAGS.model_dir, r'/home/lg/projects/frozen_mobilenet_v1_224.pb'), 'rb') as f:\r\n    graph_def = tf.GraphDef()\r\n    graph_def.ParseFromString(f.read())\r\n    _ = tf.import_graph_def(graph_def,return_elements=['MobilenetV1/Predictions/Reshape_1:0'], name='lg')\r\n\r\ndef run_inference_on_image(image):\r\n  \"\"\"Runs inference on an image.\r\n  Args:\r\n    image: Image file name.\r\n  Returns:\r\n    Nothing\r\n  \"\"\"\r\n  if not tf.gfile.Exists(image):\r\n    tf.logging.fatal('File does not exist %s', image)\r\n\r\n  image_data = tf.gfile.FastGFile(image, 'rb').read()\r\n  img_data_jpg = tf.image.decode_jpeg(image_data) #图像解码  \r\n  img_data_jpg = tf.image.convert_image_dtype(img_data_jpg, dtype=tf.float32) #改变图像数据的类型\r\n  img_data_jpg = tf.image.resize_image_with_crop_or_pad(img_data_jpg,224,224)\r\n\r\n  # Creates graph from saved GraphDef.\r\n  create_graph()\r\n\r\n  with tf.Session() as sess:\r\n    image_data = img_data_jpg.eval().reshape(-1,224,224,3)\r\n    softmax_tensor = sess.graph.get_tensor_by_name('lg/MobilenetV1/Predictions/Reshape_1:0')\r\n    predictions = sess.run(softmax_tensor, {'lg/Placeholder:0': image_data})\r\n    predictions = np.squeeze(predictions)\r\n    print('predictions: ',predictions)\r\n    # Read the labels from label.txt.\r\n    label_path = os.path.join(FLAGS.model_dir, '/home/lg/projects/labels.txt')\r\n    label = np.loadtxt(fname=label_path,dtype=str)\r\n\r\n    top_k = predictions.argsort()[-FLAGS.num_top_predictions:][::-1]\r\n    for node_id in top_k:\r\n      label_string = label[node_id]\r\n      score = predictions[node_id]\r\n      print('%s (score = %.5f)' % (label_string, score))\r\n\r\ndef main(_):\r\n  image = (FLAGS.image_file if FLAGS.image_file else os.path.join(FLAGS.model_dir, 'cropped_panda.jpg'))\r\n  run_inference_on_image(image)\r\n\r\nif __name__ == '__main__':\r\n  parser = argparse.ArgumentParser()\r\n  # graph_def.pb: Binary representation of the GraphDef protocol buffer.\r\n  # label.txt: the labels according to data tfrecord\r\n  parser.add_argument(\r\n      '--model_dir',\r\n      type=str,\r\n      default='/tmp/imagenet',\r\n      help='Path to graph_def.pb and label.txt'\r\n  )\r\n  parser.add_argument(\r\n      '--image_file',\r\n      type=str,\r\n      default=r'/home/lg/projects/data/flower_photos/daisy/5673728_71b8cb57eb.jpg',\r\n      help='Absolute path to image file.'\r\n  )\r\n  parser.add_argument(\r\n      '--num_top_predictions',\r\n      type=int,\r\n      default=2,\r\n      help='Display this many predictions.'\r\n  )\r\n  FLAGS, unparsed = parser.parse_known_args()\r\ntf.app.run(main=main, argv=[sys.argv[0]] + unparsed)\r\n```\r\n\r\nThe mobilenet v1 I used is in https://github.com/tensorflow/models/tree/master/research/slim/nets .\r\nI have tried inception v3, whether to eval or classify an image that can get right results. So I think  mobilenet_v1_eval.py must have a bug. \r\n\r\nActually, I have tried two experiments, one is the custom codes(tf_train.py, tf_eval.py, tf_input.py, tf_inference.py, tf_classify_image.py referencing https://github.com/tensorflow/models/tree/master/tutorials/image/cifar10) in https://github.com/GarryLau/draft_notes/tree/master/TF.\r\nAnother one is referencing mobilenet_v1_train.py, mobilenet_v1_eval.py, mobilenet_v1.py, in https://github.com/tensorflow/models/tree/master/research/slim/nets .\r\nIn the begining, I think the custom codes have some problems, so I tried the second experiment that tensorflow official. But they are all wrong.\r\n\r\n### Source code / logs\r\nAll the codes I used are in:\r\nhttps://github.com/GarryLau/draft_notes/tree/master/TF\r\nhttps://github.com/GarryLau/draft_notes/tree/master/create_tfrecord\r\n"},"comment":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/comments/415891947","html_url":"https://github.com/tensorflow/tensorflow/issues/18774#issuecomment-415891947","issue_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/18774","id":415891947,"node_id":"MDEyOklzc3VlQ29tbWVudDQxNTg5MTk0Nw==","user":{"login":"tensorflowbutler","id":28546240,"node_id":"MDQ6VXNlcjI4NTQ2MjQw","avatar_url":"https://avatars2.githubusercontent.com/u/28546240?v=4","gravatar_id":"","url":"https://api.github.com/users/tensorflowbutler","html_url":"https://github.com/tensorflowbutler","followers_url":"https://api.github.com/users/tensorflowbutler/followers","following_url":"https://api.github.com/users/tensorflowbutler/following{/other_user}","gists_url":"https://api.github.com/users/tensorflowbutler/gists{/gist_id}","starred_url":"https://api.github.com/users/tensorflowbutler/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tensorflowbutler/subscriptions","organizations_url":"https://api.github.com/users/tensorflowbutler/orgs","repos_url":"https://api.github.com/users/tensorflowbutler/repos","events_url":"https://api.github.com/users/tensorflowbutler/events{/privacy}","received_events_url":"https://api.github.com/users/tensorflowbutler/received_events","type":"User","site_admin":false},"created_at":"2018-08-24T21:50:48Z","updated_at":"2018-08-24T21:50:48Z","author_association":"MEMBER","body":"Nagging Assignee @tatatodd: It has been 106 days with no activity and this issue has an assignee. Please update the label and/or status accordingly."}}
{"actor":{"display_login":"tensorflowbutler"}}