{"action":"created","issue":{"url":"https://api.github.com/repos/BVLC/caffe/issues/5018","repository_url":"https://api.github.com/repos/BVLC/caffe","labels_url":"https://api.github.com/repos/BVLC/caffe/issues/5018/labels{/name}","comments_url":"https://api.github.com/repos/BVLC/caffe/issues/5018/comments","events_url":"https://api.github.com/repos/BVLC/caffe/issues/5018/events","html_url":"https://github.com/BVLC/caffe/issues/5018","id":191232269,"number":5018,"title":"OpenCL-branch, GPU mode freezes but CPU mode works","user":{"login":"kbxu","id":6727603,"avatar_url":"https://avatars.githubusercontent.com/u/6727603?v=3","gravatar_id":"","url":"https://api.github.com/users/kbxu","html_url":"https://github.com/kbxu","followers_url":"https://api.github.com/users/kbxu/followers","following_url":"https://api.github.com/users/kbxu/following{/other_user}","gists_url":"https://api.github.com/users/kbxu/gists{/gist_id}","starred_url":"https://api.github.com/users/kbxu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kbxu/subscriptions","organizations_url":"https://api.github.com/users/kbxu/orgs","repos_url":"https://api.github.com/users/kbxu/repos","events_url":"https://api.github.com/users/kbxu/events{/privacy}","received_events_url":"https://api.github.com/users/kbxu/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2016-11-23T10:19:14Z","updated_at":"2016-11-24T03:20:17Z","closed_at":"2016-11-23T11:50:33Z","body":"Dear community members,\r\n\r\nI'm using caffe-opencl in Linux Mint 17 with AMD HD7470 1G, RAM 8G. I've built it twice but with different output when running. And now the GPU mode freezes.\r\n\r\nIn the first build, I was following compilation instruction from caffe-master pages (http://caffe.berkeleyvision.org/install_apt.html)\r\nI succeeded compiling but when running on LeNet using `sh examples/mnist/train_lenet_rmsprop.sh` (I've already transformed the data into lmdb using the script),\r\nit kept popping out three below lines from `data_layer.cpp`, without loss.\r\n```\r\nprefetch time 1ms\r\ndata time 0.4ms\r\ntransform time 0.5ms\r\n```\r\n(The second build) `make runtest` shows that `NetTest/0.TestSharedWeightsUpdate` failed.\r\nLater I found the opencl compilation needs clBLAS in (https://github.com/amd/OpenCL-caffe/wiki)\r\nI followed the instruction and built clBLAS. But I then got confused of how to change BLAS/ATLAS to clBLAS using cmake. So I `rm *` for all the old built and built another time using the same parameters (still using ViennaCL as BLAS). This time when I ran I didn't get into loops of endless `prefetch time` output. However when I ran LeNet the GPU load raises to 99% in few seconds and it froze at \r\n```\r\nI1123 17:56:16.679487 14177 caffe.cpp:278] Starting Optimization\r\nI1123 17:56:16.679507 14177 solver.cpp:299] Solving LeNet\r\nI1123 17:56:16.679520 14177 solver.cpp:300] Learning Rate Policy: inv\r\nI1123 17:56:16.685703 14177 solver.cpp:358] Iteration 0, Testing net (#0)\r\n```\r\nThe CPU mode runs ok. \r\n\r\nCould be there something I neglected? Is there any suggestions for the installation? I now realized the second built instruction is from another repo (but I didn't change the configuration).\r\n\r\nBelow is the output of `caffe device_query`\r\n```\r\nI1123 18:17:56.751840 14380 common.cpp:373] Total devices: 2\r\nI1123 18:17:56.752209 14380 common.cpp:374] CUDA devices: 0\r\nI1123 18:17:56.752226 14380 common.cpp:375] OpenCL devices: 2\r\nI1123 18:17:56.752238 14380 common.cpp:399] Device id:                     0\r\nI1123 18:17:56.752250 14380 common.cpp:401] Device backend:                OpenCL\r\nI1123 18:17:56.752280 14380 common.cpp:403] Backend details:               Advanced Micro Devices, Inc.: OpenCL 2.0 AMD-APP (1598.5)\r\nI1123 18:17:56.752300 14380 common.cpp:405] Device vendor:                 Advanced Micro Devices, Inc.\r\nI1123 18:17:56.752315 14380 common.cpp:407] Name:                          Caicos\r\nI1123 18:17:56.752328 14380 common.cpp:409] Total global memory:           536870912\r\nI1123 18:17:56.752341 14380 common.cpp:399] Device id:                     1\r\nI1123 18:17:56.752352 14380 common.cpp:401] Device backend:                OpenCL\r\nI1123 18:17:56.752367 14380 common.cpp:403] Backend details:               Advanced Micro Devices, Inc.: OpenCL 2.0 AMD-APP (1598.5)\r\nI1123 18:17:56.752379 14380 common.cpp:405] Device vendor:                 GenuineIntel\r\nI1123 18:17:56.752391 14380 common.cpp:407] Name:                          Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz\r\nI1123 18:17:56.752403 14380 common.cpp:409] Total global memory:           8325865472\r\n\r\n```\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/BVLC/caffe/issues/comments/262681501","html_url":"https://github.com/BVLC/caffe/issues/5018#issuecomment-262681501","issue_url":"https://api.github.com/repos/BVLC/caffe/issues/5018","id":262681501,"user":{"login":"kbxu","id":6727603,"avatar_url":"https://avatars.githubusercontent.com/u/6727603?v=3","gravatar_id":"","url":"https://api.github.com/users/kbxu","html_url":"https://github.com/kbxu","followers_url":"https://api.github.com/users/kbxu/followers","following_url":"https://api.github.com/users/kbxu/following{/other_user}","gists_url":"https://api.github.com/users/kbxu/gists{/gist_id}","starred_url":"https://api.github.com/users/kbxu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kbxu/subscriptions","organizations_url":"https://api.github.com/users/kbxu/orgs","repos_url":"https://api.github.com/users/kbxu/repos","events_url":"https://api.github.com/users/kbxu/events{/privacy}","received_events_url":"https://api.github.com/users/kbxu/received_events","type":"User","site_admin":false},"created_at":"2016-11-24T03:20:17Z","updated_at":"2016-11-24T03:20:17Z","body":"Also, the GPU seems to be slower than CPU in 100 iterations. (in the no libDNN build)\r\n```\r\nI1124 11:17:48.136464 14473 common.cpp:373] Total devices: 2\r\nI1124 11:17:48.136653 14473 common.cpp:374] CUDA devices: 0\r\nI1124 11:17:48.136667 14473 common.cpp:375] OpenCL devices: 2\r\nI1124 11:17:48.136674 14473 common.cpp:399] Device id:                     0\r\nI1124 11:17:48.136684 14473 common.cpp:401] Device backend:                OpenCL\r\nI1124 11:17:48.136701 14473 common.cpp:403] Backend details:               Advanced Micro Devices, Inc.: OpenCL 2.0 AMD-APP (1598.5)\r\nI1124 11:17:48.136714 14473 common.cpp:405] Device vendor:                 Advanced Micro Devices, Inc.\r\nI1124 11:17:48.136725 14473 common.cpp:407] Name:                          Caicos\r\nI1124 11:17:48.136735 14473 common.cpp:409] Total global memory:           536870912\r\nI1124 11:17:48.136745 14473 common.cpp:399] Device id:                     1\r\nI1124 11:17:48.136754 14473 common.cpp:401] Device backend:                OpenCL\r\nI1124 11:17:48.136765 14473 common.cpp:403] Backend details:               Advanced Micro Devices, Inc.: OpenCL 2.0 AMD-APP (1598.5)\r\nI1124 11:17:48.136773 14473 common.cpp:405] Device vendor:                 GenuineIntel\r\nI1124 11:17:48.136782 14473 common.cpp:407] Name:                          Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz\r\nI1124 11:17:48.136791 14473 common.cpp:409] Total global memory:           8325865472\r\n\r\ncaffe time -model=examples/mnist/lenet_train_test.prototxt -iterations 100 -gpu 1\r\n...\r\nI1124 11:16:08.494714 14445 caffe.cpp:465] Average Forward pass: 68.3438 ms.\r\nI1124 11:16:08.494748 14445 caffe.cpp:467] Average Backward pass: 100.068 ms.\r\nI1124 11:16:08.494762 14445 caffe.cpp:469] Average Forward-Backward: 168.602 ms.\r\nI1124 11:16:08.494776 14445 caffe.cpp:471] Total Time: 16860.2 ms.\r\nI1124 11:16:08.494789 14445 caffe.cpp:472] *** Benchmark ends ***\r\n\r\ncaffe time -model=examples/mnist/lenet_train_test.prototxt -iterations 100 -gpu 0\r\n...\r\nI1124 11:17:13.445297 14455 caffe.cpp:465] Average Forward pass: 152.661 ms.\r\nI1124 11:17:13.445358 14455 caffe.cpp:467] Average Backward pass: 123.195 ms.\r\nI1124 11:17:13.445390 14455 caffe.cpp:469] Average Forward-Backward: 276.549 ms.\r\nI1124 11:17:13.445415 14455 caffe.cpp:471] Total Time: 27654.9 ms.\r\nI1124 11:17:13.445439 14455 caffe.cpp:472] *** Benchmark ends ***\r\n```"}}
{"actor":{"display_login":"kbxu"}}