{"action":"created","issue":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/18943","repository_url":"https://api.github.com/repos/tensorflow/tensorflow","labels_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/18943/labels{/name}","comments_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/18943/comments","events_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/18943/events","html_url":"https://github.com/tensorflow/tensorflow/issues/18943","id":318602490,"number":18943,"title":"ssd_mobilenet_v2 was slower than ssd_mobilenet_v1 in the tflite ?","user":{"login":"WenguoLi","id":31765154,"avatar_url":"https://avatars0.githubusercontent.com/u/31765154?v=4","gravatar_id":"","url":"https://api.github.com/users/WenguoLi","html_url":"https://github.com/WenguoLi","followers_url":"https://api.github.com/users/WenguoLi/followers","following_url":"https://api.github.com/users/WenguoLi/following{/other_user}","gists_url":"https://api.github.com/users/WenguoLi/gists{/gist_id}","starred_url":"https://api.github.com/users/WenguoLi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/WenguoLi/subscriptions","organizations_url":"https://api.github.com/users/WenguoLi/orgs","repos_url":"https://api.github.com/users/WenguoLi/repos","events_url":"https://api.github.com/users/WenguoLi/events{/privacy}","received_events_url":"https://api.github.com/users/WenguoLi/received_events","type":"User","site_admin":false},"labels":[{"id":386191887,"url":"https://api.github.com/repos/tensorflow/tensorflow/labels/stat:awaiting%20response","name":"stat:awaiting response","color":"5319e7","default":false}],"state":"open","locked":false,"assignee":{"login":"shivaniag","id":16565716,"avatar_url":"https://avatars1.githubusercontent.com/u/16565716?v=4","gravatar_id":"","url":"https://api.github.com/users/shivaniag","html_url":"https://github.com/shivaniag","followers_url":"https://api.github.com/users/shivaniag/followers","following_url":"https://api.github.com/users/shivaniag/following{/other_user}","gists_url":"https://api.github.com/users/shivaniag/gists{/gist_id}","starred_url":"https://api.github.com/users/shivaniag/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shivaniag/subscriptions","organizations_url":"https://api.github.com/users/shivaniag/orgs","repos_url":"https://api.github.com/users/shivaniag/repos","events_url":"https://api.github.com/users/shivaniag/events{/privacy}","received_events_url":"https://api.github.com/users/shivaniag/received_events","type":"User","site_admin":false},"assignees":[{"login":"shivaniag","id":16565716,"avatar_url":"https://avatars1.githubusercontent.com/u/16565716?v=4","gravatar_id":"","url":"https://api.github.com/users/shivaniag","html_url":"https://github.com/shivaniag","followers_url":"https://api.github.com/users/shivaniag/followers","following_url":"https://api.github.com/users/shivaniag/following{/other_user}","gists_url":"https://api.github.com/users/shivaniag/gists{/gist_id}","starred_url":"https://api.github.com/users/shivaniag/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shivaniag/subscriptions","organizations_url":"https://api.github.com/users/shivaniag/orgs","repos_url":"https://api.github.com/users/shivaniag/repos","events_url":"https://api.github.com/users/shivaniag/events{/privacy}","received_events_url":"https://api.github.com/users/shivaniag/received_events","type":"User","site_admin":false}],"milestone":null,"comments":3,"created_at":"2018-04-28T05:34:07Z","updated_at":"2018-05-02T02:36:08Z","closed_at":null,"author_association":"NONE","body":"### System information\r\n- **Have I written custom code (as opposed to using a stock example script provided in TensorFlow)**:no\r\n- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**:Ubuntu 16.04\r\n- **TensorFlow installed from (source or binary)**:Source\r\n- **TensorFlow version (use command below)**:1.8.0rc0\r\n- **Python version**: 2.7.12\r\n- **Bazel version (if compiling from source)**: 0.12.0\r\n- **GCC/Compiler version (if compiling from source)**: 5.4.0\r\n- **CUDA/cuDNN version**:cuda-9.0/7.0\r\n- **GPU model and memory**:GeForce GTX 1080/8105MiB\r\n- **Phone**: xiaomi5 (Snapdragon 820)\r\n\r\n### Describe the problem\r\nUsing tflite's [benchmark_model](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/tools/benchmark_model.cc) tool, I tested the performance of the ssd_mobilenet_v1 and ssd_mobilenet_v2  models , and found that ssd_mobilenet_v2 was slower than ssd_mobilenet_v1, about 10ms, when setting to 4 threads.\r\n\r\nBut SSDLite presented in [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://arxiv.org/abs/1801.04381) is 35% faster than Mobilenet V1 SSD on a Google Pixel phone CPU (200ms vs. 270ms) at the same accuracy.\r\n\r\nWho can explain why ?\r\n### Source code / logs\r\nssd_mobilenet_v1 : \r\n```\r\n ./benchmark_model \\\r\n>   --graph=\"mobilenet_ssd_v1_quan.tflite\" \\\r\n>   --input_layer=\"Preprocessor/sub\" \\\r\n>   --input_layer_shape=\"1,300,300,3\" \\\r\n>   --input_layer_type=\"uint8\" \\\r\n>   --run_delay=\"-1.0\" \\\r\n>   --output_layer=\"concat,concat_1\" \\\r\n>   --num_runs=50 \\\r\n>   --num_threads=4 \\\r\n>   --warmup_runs=1 \\\r\n>   --use_nnapi=false\r\nWARNING: linker: /data/local/tmp/benchmark_model: unused DT entry: type 0xf arg 0x82e\r\nGraph: [mobilenet_ssd_v1_quan.tflite]\r\nInput layers: [Preprocessor/sub]\r\nInput shapes: [1,300,300,3]\r\nInput types: [uint8]\r\nOutput layers: [concat,concat_1]\r\nNum runs: [50]\r\nInter-run delay (seconds): [-1.0]\r\nNum threads: [4]\r\nWarmup runs: [1]\r\nUse nnapi : [0]\r\nEnable profiling : [0]\r\nnnapi error: unable to open library libneuralnetworks.so\r\nInitialized session in 0.02646s\r\nRunning benchmark for 1 iterations: \r\ncount=1 min=220752 max=220752 avg=220752 std=0\r\nRunning benchmark for 50 iterations: \r\ncount=50 min=75066 max=193842 avg=82848.9 std=19942\r\nAverage inference timings in us: 82848 , Warmup: 220752,\r\n``` \r\nssd_mobilenet_v2 :\r\n```\r\n./benchmark_model \\\r\n>   --graph=\"mobilenet_ssd_v2_quan.tflite\" \\\r\n>   --input_layer=\"Preprocessor/sub\" \\\r\n>   --input_layer_shape=\"1,300,300,3\" \\\r\n>   --input_layer_type=\"uint8\" \\\r\n>   --run_delay=\"-1.0\" \\\r\n>   --output_layer=\"concat,concat_1\" \\\r\n>   --num_runs=50 \\\r\n>   --num_threads=4 \\\r\n>   --warmup_runs=1 \\\r\n>   --use_nnapi=false\r\nWARNING: linker: /data/local/tmp/benchmark_model: unused DT entry: type 0xf arg 0x82e\r\nGraph: [mobilenet_ssd_v2_quan.tflite]\r\nInput layers: [Preprocessor/sub]\r\nInput shapes: [1,300,300,3]\r\nInput types: [uint8]\r\nOutput layers: [concat,concat_1]\r\nNum runs: [50]\r\nInter-run delay (seconds): [-1.0]\r\nNum threads: [4]\r\nWarmup runs: [1]\r\nUse nnapi : [0]\r\nEnable profiling : [0]\r\nnnapi error: unable to open library libneuralnetworks.so\r\nInitialized session in 0.033135s\r\nRunning benchmark for 1 iterations: \r\ncount=1 min=110434 max=110434 avg=110434 std=0\r\nRunning benchmark for 50 iterations: \r\ncount=50 min=78189 max=285788 avg=90162.9 std=29644\r\nAverage inference timings in us: 90162 , Warmup: 110434, \r\n```"},"comment":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/comments/385847873","html_url":"https://github.com/tensorflow/tensorflow/issues/18943#issuecomment-385847873","issue_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/18943","id":385847873,"user":{"login":"WenguoLi","id":31765154,"avatar_url":"https://avatars0.githubusercontent.com/u/31765154?v=4","gravatar_id":"","url":"https://api.github.com/users/WenguoLi","html_url":"https://github.com/WenguoLi","followers_url":"https://api.github.com/users/WenguoLi/followers","following_url":"https://api.github.com/users/WenguoLi/following{/other_user}","gists_url":"https://api.github.com/users/WenguoLi/gists{/gist_id}","starred_url":"https://api.github.com/users/WenguoLi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/WenguoLi/subscriptions","organizations_url":"https://api.github.com/users/WenguoLi/orgs","repos_url":"https://api.github.com/users/WenguoLi/repos","events_url":"https://api.github.com/users/WenguoLi/events{/privacy}","received_events_url":"https://api.github.com/users/WenguoLi/received_events","type":"User","site_admin":false},"created_at":"2018-05-02T02:36:08Z","updated_at":"2018-05-02T02:36:08Z","author_association":"NONE","body":"@Davidnet , ssd_mobilenet_v1 and ssd_mobilenet_v2 models are running on android mobile devices, not on desktop GPU.\r\nI found the reason why kernel size of box_predictor is 1 in the ssd_mobilenet_v1. but kernel size of box_predictor is 3 in the ssd_mobilenet_v2."}}
{"actor":{"display_login":"WenguoLi"}}