{"action":"created","issue":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/21761","repository_url":"https://api.github.com/repos/tensorflow/tensorflow","labels_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/21761/labels{/name}","comments_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/21761/comments","events_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/21761/events","html_url":"https://github.com/tensorflow/tensorflow/issues/21761","id":352566030,"node_id":"MDU6SXNzdWUzNTI1NjYwMzA=","number":21761,"title":"[XLA] code generation for ARM NEON produce very slow code with tensorflow 1.9.0 and 1.10.0","user":{"login":"guillaume-michel","id":935309,"node_id":"MDQ6VXNlcjkzNTMwOQ==","avatar_url":"https://avatars0.githubusercontent.com/u/935309?v=4","gravatar_id":"","url":"https://api.github.com/users/guillaume-michel","html_url":"https://github.com/guillaume-michel","followers_url":"https://api.github.com/users/guillaume-michel/followers","following_url":"https://api.github.com/users/guillaume-michel/following{/other_user}","gists_url":"https://api.github.com/users/guillaume-michel/gists{/gist_id}","starred_url":"https://api.github.com/users/guillaume-michel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/guillaume-michel/subscriptions","organizations_url":"https://api.github.com/users/guillaume-michel/orgs","repos_url":"https://api.github.com/users/guillaume-michel/repos","events_url":"https://api.github.com/users/guillaume-michel/events{/privacy}","received_events_url":"https://api.github.com/users/guillaume-michel/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"sanjoy","id":136291,"node_id":"MDQ6VXNlcjEzNjI5MQ==","avatar_url":"https://avatars2.githubusercontent.com/u/136291?v=4","gravatar_id":"","url":"https://api.github.com/users/sanjoy","html_url":"https://github.com/sanjoy","followers_url":"https://api.github.com/users/sanjoy/followers","following_url":"https://api.github.com/users/sanjoy/following{/other_user}","gists_url":"https://api.github.com/users/sanjoy/gists{/gist_id}","starred_url":"https://api.github.com/users/sanjoy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sanjoy/subscriptions","organizations_url":"https://api.github.com/users/sanjoy/orgs","repos_url":"https://api.github.com/users/sanjoy/repos","events_url":"https://api.github.com/users/sanjoy/events{/privacy}","received_events_url":"https://api.github.com/users/sanjoy/received_events","type":"User","site_admin":false},"assignees":[{"login":"sanjoy","id":136291,"node_id":"MDQ6VXNlcjEzNjI5MQ==","avatar_url":"https://avatars2.githubusercontent.com/u/136291?v=4","gravatar_id":"","url":"https://api.github.com/users/sanjoy","html_url":"https://github.com/sanjoy","followers_url":"https://api.github.com/users/sanjoy/followers","following_url":"https://api.github.com/users/sanjoy/following{/other_user}","gists_url":"https://api.github.com/users/sanjoy/gists{/gist_id}","starred_url":"https://api.github.com/users/sanjoy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sanjoy/subscriptions","organizations_url":"https://api.github.com/users/sanjoy/orgs","repos_url":"https://api.github.com/users/sanjoy/repos","events_url":"https://api.github.com/users/sanjoy/events{/privacy}","received_events_url":"https://api.github.com/users/sanjoy/received_events","type":"User","site_admin":false},{"login":"hawkinsp","id":348932,"node_id":"MDQ6VXNlcjM0ODkzMg==","avatar_url":"https://avatars0.githubusercontent.com/u/348932?v=4","gravatar_id":"","url":"https://api.github.com/users/hawkinsp","html_url":"https://github.com/hawkinsp","followers_url":"https://api.github.com/users/hawkinsp/followers","following_url":"https://api.github.com/users/hawkinsp/following{/other_user}","gists_url":"https://api.github.com/users/hawkinsp/gists{/gist_id}","starred_url":"https://api.github.com/users/hawkinsp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hawkinsp/subscriptions","organizations_url":"https://api.github.com/users/hawkinsp/orgs","repos_url":"https://api.github.com/users/hawkinsp/repos","events_url":"https://api.github.com/users/hawkinsp/events{/privacy}","received_events_url":"https://api.github.com/users/hawkinsp/received_events","type":"User","site_admin":false}],"milestone":null,"comments":2,"created_at":"2018-08-21T14:25:45Z","updated_at":"2018-08-30T23:43:14Z","closed_at":null,"author_association":"NONE","body":"### System information\r\n- **Have I written custom code (as opposed to using a stock example script provided in TensorFlow)**: NO\r\n- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: Linux Ubuntu 14.04\r\n- **Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device**: Cortex A7\r\n- **TensorFlow installed from (source or binary)**: from source\r\n- **TensorFlow version (use command below)**: v1.9.0 and superior\r\n- **Python version**: 3.4\r\n- **Bazel version (if compiling from source)**: 0.15.2\r\n- **GCC/Compiler version (if compiling from source)**: 4.9.4\r\n- **CUDA/cuDNN version**: N/A\r\n- **GPU model and memory**: N/A\r\n- **Exact command to reproduce**:\r\ntfcompile --graph=network.pb --config=network.config.pbtxt  --cpp_class=Network --out_header=network.h --out_function_object=network.o --xla_enable_fast_math=true --target_triple=armv7a-none-android --target_features=+neon --entry_point=run_network\r\n\r\narm-linux-androideabi-g++ -shared -o test.so network.o\r\n\r\n(details of network.pb and network.config.pbtxt are not relevant, The only thing I can say is that it contains 2D Convolution and Dense layer)\r\n\r\n### Describe the problem\r\nI use XLA AOT (tfcompile) to compile network down to shared library with tfcompile and it was working great until v1.9.0.\r\nStarting from v1.9.0, I see huge speed penalty (more than 15x slowdown) when executing the network on ARM v7 + NEON. After some investigation, **I think it is pretty clear that v1.9.0 and v1.10.0 do not use specialized neon runtime for matmul and conv** (even when +neon is specified in target_features) as shown in the following objdumps:\r\n\r\n- ** With tensorflow v1.8.0 and previous versions\r\nobjdump -CT test.so               \r\n\r\ntest.so:     file format elf32-little\r\n\r\nDYNAMIC SYMBOL TABLE:\r\n00000000      DF *UND*\t00000000  LIBC        __cxa_finalize\r\n00000000      DF *UND*\t00000000  LIBC        __cxa_atexit\r\n00000000      D  *UND*\t00000000              **__xla_cpu_runtime_EigenConvF32**\r\n00000000      D  *UND*\t00000000              **__xla_cpu_runtime_EigenMatMulF32**\r\n000003c0 g    DF .text\t0000abc0  Base        run_network\r\n00672004 g    D  *ABS*\t00000000  Base        __bss_start\r\n00672004 g    D  *ABS*\t00000000  Base        _end\r\n00672004 g    D  *ABS*\t00000000  Base        _edata\r\n\r\n- ** With tensorflow v1.9.0 and superior:\r\n\r\nobjdump -CT test.so \r\n\r\ntest.so:     file format elf32-little\r\n\r\nDYNAMIC SYMBOL TABLE:\r\n00000000      DF *UND*\t00000000  LIBC        __cxa_finalize\r\n00000000      DF *UND*\t00000000  LIBC        __cxa_atexit\r\n00000320 g    DF .text\t0000cb40  Base        run_network\r\n00674004 g    D  *ABS*\t00000000  Base        __bss_start\r\n00674004 g    D  *ABS*\t00000000  Base        _end\r\n00674004 g    D  *ABS*\t00000000  Base        _edata\r\n\r\nI tried to follow the code generation process but got lost in LLVM code generation.\r\nCould you have a look at the problem, please? Any help will be helpful.\r\n\r\n### Source code / logs\r\nRunning code just call run_network function in the shared library with the correct parameters."},"comment":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/comments/417502873","html_url":"https://github.com/tensorflow/tensorflow/issues/21761#issuecomment-417502873","issue_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/21761","id":417502873,"node_id":"MDEyOklzc3VlQ29tbWVudDQxNzUwMjg3Mw==","user":{"login":"sanjoy","id":136291,"node_id":"MDQ6VXNlcjEzNjI5MQ==","avatar_url":"https://avatars2.githubusercontent.com/u/136291?v=4","gravatar_id":"","url":"https://api.github.com/users/sanjoy","html_url":"https://github.com/sanjoy","followers_url":"https://api.github.com/users/sanjoy/followers","following_url":"https://api.github.com/users/sanjoy/following{/other_user}","gists_url":"https://api.github.com/users/sanjoy/gists{/gist_id}","starred_url":"https://api.github.com/users/sanjoy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sanjoy/subscriptions","organizations_url":"https://api.github.com/users/sanjoy/orgs","repos_url":"https://api.github.com/users/sanjoy/repos","events_url":"https://api.github.com/users/sanjoy/events{/privacy}","received_events_url":"https://api.github.com/users/sanjoy/received_events","type":"User","site_admin":false},"created_at":"2018-08-30T23:43:14Z","updated_at":"2018-08-30T23:43:14Z","author_association":"MEMBER","body":"Can you please attach `network.pb`  and `network.config.pbtxt` so that I can try reproducing this?"}}
{"actor":{"display_login":"sanjoy"}}