{"action":"created","issue":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/19277","repository_url":"https://api.github.com/repos/tensorflow/tensorflow","labels_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/19277/labels{/name}","comments_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/19277/comments","events_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/19277/events","html_url":"https://github.com/tensorflow/tensorflow/issues/19277","id":322949394,"node_id":"MDU6SXNzdWUzMjI5NDkzOTQ=","number":19277,"title":"SSD mobilenet inference is slower w/ MKL","user":{"login":"YijinLiu","id":10109708,"node_id":"MDQ6VXNlcjEwMTA5NzA4","avatar_url":"https://avatars1.githubusercontent.com/u/10109708?v=4","gravatar_id":"","url":"https://api.github.com/users/YijinLiu","html_url":"https://github.com/YijinLiu","followers_url":"https://api.github.com/users/YijinLiu/followers","following_url":"https://api.github.com/users/YijinLiu/following{/other_user}","gists_url":"https://api.github.com/users/YijinLiu/gists{/gist_id}","starred_url":"https://api.github.com/users/YijinLiu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/YijinLiu/subscriptions","organizations_url":"https://api.github.com/users/YijinLiu/orgs","repos_url":"https://api.github.com/users/YijinLiu/repos","events_url":"https://api.github.com/users/YijinLiu/events{/privacy}","received_events_url":"https://api.github.com/users/YijinLiu/received_events","type":"User","site_admin":false},"labels":[{"id":404586558,"node_id":"MDU6TGFiZWw0MDQ1ODY1NTg=","url":"https://api.github.com/repos/tensorflow/tensorflow/labels/stat:community%20support","name":"stat:community support","color":"c2e0c6","default":false},{"id":473172988,"node_id":"MDU6TGFiZWw0NzMxNzI5ODg=","url":"https://api.github.com/repos/tensorflow/tensorflow/labels/type:bug/performance","name":"type:bug/performance","color":"fbca04","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":9,"created_at":"2018-05-14T19:29:07Z","updated_at":"2018-10-08T17:44:21Z","closed_at":null,"author_association":"NONE","body":"### System information\r\n- **Have I written custom code (as opposed to using a stock example script provided in TensorFlow)**:\r\nNo.\r\n- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**:\r\nLinux Ubuntu 16.04\r\n- **TensorFlow installed from (source or binary)**:\r\nSource\r\n- **TensorFlow version (use command below)**:\r\n1.8.0 (could repro the same problem from head too)\r\n- **Python version**: \r\nN/A\r\n- **Bazel version (if compiling from source)**:\r\n0.13.0\r\n- **GCC/Compiler version (if compiling from source)**:\r\n5.4.0\r\n- **CUDA/cuDNN version**:\r\nN/A\r\n- **GPU model and memory**:\r\nN/A\r\n- **Exact command to reproduce**:\r\nOMP_NUM_THREADS=1 bazel run --config=mkl --config=opt --config=monolithic //tensorflow/tools/benchmark:benchmark_model -- --graph=ssd_mobilenet_v2_coco_2018_03_29_frozen.pb --show_flops --input_layer=image_tensor --input_layer_type=uint8 --input_layer_shape=1,1920,1080,3 --output_layer=num_detections,detection_classes,detection_scores,detection_boxes --num_threads=1\r\n\r\n### Describe the problem\r\nw/ MKL, benchmark_model got 18.98B FLOPs/second, w/o MKL, it got 25.61B.\r\nFrom the benchmark_model results, we could see that _MklConv2DWithBias is the culprit.\r\nI am using MKL 2018.2.199 and mkldnn 0.14 on a i7-5557U CPU.\r\n\r\nA unrelated question: @agramesh1 I filed the same bug on [mkldnn](https://github.com/intel/mkl-dnn/issues/234). They told me you have some plan to implement MKL version of DepthwiseConv2dNative. Do you have a timeline for it? I am eager to try it..\r\n\r\nbenchmark_model results:\r\n\r\n w/ MKL\r\n\r\n             [Node type]  [count]  [avg ms]    [avg %]    [cdf %]  [mem KB][times called]\r\n       _MklConv2DWithBias       12   143.936    42.710%    42.710% 56555.531       12\r\n               _MklConv2D       43    48.872    14.502%    57.212% 43206.305       43\r\n      DepthwiseConv2dNative       21    27.330     8.110%    65.321% 17180.992       21\r\n                  _MklMul       43    12.751     3.784%    69.105% 32070.080       43\r\n                  _MklAdd       53    12.137     3.601%    72.706% 33662.238       53\r\n                     Cast      183    11.584     3.437%    76.143% 24883.217      183\r\n        _MklInputConversion       96    10.990     3.261%    79.404% 32261.969       96\r\n                    Const     1602    10.360     3.074%    82.479%     0.000     1602\r\n                      Mul      127     7.063     2.096%    84.574%     0.004      127\r\n                      Add      119     6.541     1.941%    86.515%    15.344      119\r\n                    Relu6       47     3.715     1.102%    87.618%     0.000       47\r\n                  Minimum      451     3.680     1.092%    88.709%     0.000      451\r\n                   Gather      546     3.607     1.070%    89.780%     0.000      546\r\n                    Slice       93     3.543     1.051%    90.831%  1380.240       93\r\n       TensorArrayScatterV3        5     3.261     0.968%    91.799% 25604.012        5\r\n              _MklReshape      107     3.183     0.944%    92.743%   810.972      107\r\n                    Where      180     2.560     0.760%    93.503%     1.440      180\r\n             _MklConcatV2       98     2.260     0.671%    94.173%   782.604       98\r\n                  Greater      183     2.183     0.648%    94.821%   172.533      183\r\n\r\nw/o MKL\r\n\r\n              [Node type]  [count]  [avg ms]    [avg %]    [cdf %]  [mem KB][times called]\r\n                   Conv2D       55   108.692    47.515%    47.515% 32798.539       55\r\n    DepthwiseConv2dNative       21    28.022    12.250%    59.765% 17180.992       21\r\n                      Mul      170    18.866     8.247%    68.012%     0.008      170\r\n                      Add      172    18.460     8.070%    76.082%    15.344      172\r\n                     Cast      183    12.116     5.297%    81.378% 24883.217      183\r\n                   Gather      546     4.815     2.105%    83.483%     0.000      546\r\n     TensorArrayScatterV3        5     3.971     1.736%    85.219% 25604.012        5\r\n                    Relu6       47     3.619     1.582%    86.801%     0.000       47\r\n                    Slice       93     2.915     1.274%    88.075%  1380.240       93\r\n                  Minimum      451     2.877     1.258%    89.333%     0.000      451\r\n                    Const      476     2.802     1.225%    90.558%     0.000      476\r\n                  Maximum      360     2.761     1.207%    91.765%     0.000      360\r\n                  Reshape      287     2.318     1.013%    92.778%     0.000      287\r\n                    Where      180     1.793     0.784%    93.562%     1.440      180\r\n                      Sub      190     1.749     0.765%    94.327%     0.008      190\r\n                    Split      180     1.749     0.765%    95.091%     0.000      180\r\n                  Greater      183     1.220     0.533%    95.625%   172.533      183\r\n                 ConcatV2       99     1.057     0.462%    96.087%   730.868       99\r\n                    Shape      112     1.001     0.438%    96.524%     0.952      112\r\n                  Squeeze       92     0.978     0.428%    96.952%     0.000       92\r\n           ResizeBilinear        1     0.885     0.387%    97.339%  1080.000        1\r\n             StridedSlice      113     0.878     0.384%    97.722%     0.432      113\r\n      NonMaxSuppressionV2       90     0.820     0.358%    98.081%     0.000       90\r\n                Transpose        3     0.706     0.309%    98.390%    92.016        3\r\n                    Enter       26     0.434     0.190%    98.579%     0.000       26\r\n                     Pack       19     0.433     0.189%    98.769%    30.908       19\r\n                ZerosLike       92     0.362     0.158%    98.927%     0.004       92\r\n                  BiasAdd       12     0.326     0.143%    99.069%     0.000       12\r\n            NextIteration        8     0.308     0.135%    99.204%     0.000        8\r\n\r\n\r\n### Source code / logs\r\nI run benchmark_model with MKLDDN_VERBOSE=1 and got this log.\r\nhttps://drive.google.com/file/d/12ClzFKiOryge6So-trXrvEyEk6ycOheY/view?usp=sharing"},"comment":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/comments/427921215","html_url":"https://github.com/tensorflow/tensorflow/issues/19277#issuecomment-427921215","issue_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/19277","id":427921215,"node_id":"MDEyOklzc3VlQ29tbWVudDQyNzkyMTIxNQ==","user":{"login":"YijinLiu","id":10109708,"node_id":"MDQ6VXNlcjEwMTA5NzA4","avatar_url":"https://avatars1.githubusercontent.com/u/10109708?v=4","gravatar_id":"","url":"https://api.github.com/users/YijinLiu","html_url":"https://github.com/YijinLiu","followers_url":"https://api.github.com/users/YijinLiu/followers","following_url":"https://api.github.com/users/YijinLiu/following{/other_user}","gists_url":"https://api.github.com/users/YijinLiu/gists{/gist_id}","starred_url":"https://api.github.com/users/YijinLiu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/YijinLiu/subscriptions","organizations_url":"https://api.github.com/users/YijinLiu/orgs","repos_url":"https://api.github.com/users/YijinLiu/repos","events_url":"https://api.github.com/users/YijinLiu/events{/privacy}","received_events_url":"https://api.github.com/users/YijinLiu/received_events","type":"User","site_admin":false},"created_at":"2018-10-08T17:44:21Z","updated_at":"2018-10-08T17:44:21Z","author_association":"NONE","body":"@agramesh1 Thanks for the update. I'll find some time to try the two PRs. Do you have numbers about how much it'll improve SSD inference compared to not using MKL?"}}
{"actor":{"display_login":"YijinLiu"}}