{"action":"created","issue":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/4434","repository_url":"https://api.github.com/repos/tensorflow/tensorflow","labels_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/4434/labels{/name}","comments_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/4434/comments","events_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/4434/events","html_url":"https://github.com/tensorflow/tensorflow/issues/4434","id":177633836,"number":4434,"title":"Why is quantized graph inference takes much more time than using the original graph?","user":{"login":"yossibiton","id":6518016,"avatar_url":"https://avatars.githubusercontent.com/u/6518016?v=3","gravatar_id":"","url":"https://api.github.com/users/yossibiton","html_url":"https://github.com/yossibiton","followers_url":"https://api.github.com/users/yossibiton/followers","following_url":"https://api.github.com/users/yossibiton/following{/other_user}","gists_url":"https://api.github.com/users/yossibiton/gists{/gist_id}","starred_url":"https://api.github.com/users/yossibiton/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yossibiton/subscriptions","organizations_url":"https://api.github.com/users/yossibiton/orgs","repos_url":"https://api.github.com/users/yossibiton/repos","events_url":"https://api.github.com/users/yossibiton/events{/privacy}","received_events_url":"https://api.github.com/users/yossibiton/received_events","type":"User","site_admin":false},"labels":[{"id":404586594,"url":"https://api.github.com/repos/tensorflow/tensorflow/labels/awaiting%20googler","name":"awaiting googler","color":"b60205","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":8,"created_at":"2016-09-18T06:45:30Z","updated_at":"2016-12-15T02:22:32Z","closed_at":null,"body":"I followed this [tutorial](https://www.tensorflow.org/versions/r0.10/how_tos/quantization/index.html) in order to quantize my graph into 8 bit.I can't share the exact graph here but i can say it's a simple convolutional neural network.\n\nWhen i run the [benchmark tool](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/tools/benchmark) over the original and quantized networks it's clear that the quantized network is much much slower (100 ms vs. 4.5 ms).\n\nSlowest nodes in original network :\n\n```\ntime average [ms]   [%] [cdf%]  [Op]    [Name]\n1.198   26.54%  26.54%  MatMul  fc10/fc10/MatMul\n0.337   7.47%   34.02%  Conv2D  conv2/Conv2D\n0.332   7.36%   41.37%  Conv2D  conv4/Conv2D\n0.323   7.15%   48.53%  Conv2D  conv3/Conv2D\n0.322   7.14%   55.66%  Conv2D  conv5/Conv2D\n0.310   6.86%   62.53%  Conv2D  conv1/Conv2D\n0.118   2.61%   65.13%  Conv2D  conv2_1/Conv2D\n0.105   2.32%   67.45%  MaxPool pool1\n```\n\nSlowest nodes in quantized network :\n\n```\ntime average [ms]   [%] [cdf%]  [Op]    [Name]\n8.289   47.67%  47.67%  QuantizedMatMul fc10/fc10/MatMul_eightbit_quantized_bias_add\n5.398   5.33%   53.00%  QuantizedConv2D conv5/Conv2D_eightbit_quantized_conv\n5.248   5.18%   58.18%  QuantizedConv2D conv4/Conv2D_eightbit_quantized_conv\n4.981   4.92%   63.10%  QuantizedConv2D conv2/Conv2D_eightbit_quantized_conv\n4.908   4.85%   67.95%  QuantizedConv2D conv3/Conv2D_eightbit_quantized_conv\n3.167   3.13%   71.07%  QuantizedConv2D conv5_1/Conv2D_eightbit_quantized_conv\n3.049   3.01%   74.08%  QuantizedConv2D conv4_1/Conv2D_eightbit_quantized_conv\n2.973   2.94%   77.02%  QuantizedMatMul fc11/MatMul_eightbit_quantized_bias_add\n```\n\nWhat is the reason for that ? \nIs it the expected behavior for quantized network ?\n### Environment info\n\nOperating System: Ubuntu 16.04\nInstalled from source, without GPU support :\n1. commit hash = 37256f4\n2. bazel version = \nBuild label: 0.3.1\nBuild target: bazel-out/local-fastbuild/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar\nBuild time: Fri Jul 29 09:09:52 2016 (1469783392)\nBuild timestamp: 1469783392\nBuild timestamp as int: 1469783392\n"},"comment":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/comments/267218785","html_url":"https://github.com/tensorflow/tensorflow/issues/4434#issuecomment-267218785","issue_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/4434","id":267218785,"user":{"login":"austingg","id":3223849,"avatar_url":"https://avatars.githubusercontent.com/u/3223849?v=3","gravatar_id":"","url":"https://api.github.com/users/austingg","html_url":"https://github.com/austingg","followers_url":"https://api.github.com/users/austingg/followers","following_url":"https://api.github.com/users/austingg/following{/other_user}","gists_url":"https://api.github.com/users/austingg/gists{/gist_id}","starred_url":"https://api.github.com/users/austingg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/austingg/subscriptions","organizations_url":"https://api.github.com/users/austingg/orgs","repos_url":"https://api.github.com/users/austingg/repos","events_url":"https://api.github.com/users/austingg/events{/privacy}","received_events_url":"https://api.github.com/users/austingg/received_events","type":"User","site_admin":false},"created_at":"2016-12-15T02:22:32Z","updated_at":"2016-12-15T02:22:32Z","body":"@chenliu0831 great. Metal use gpu, but support only limited operator. "}}
{"actor":{"display_login":"austingg"}}