{"action":"created","issue":{"url":"https://api.github.com/repos/apache/incubator-systemml/issues/307","repository_url":"https://api.github.com/repos/apache/incubator-systemml","labels_url":"https://api.github.com/repos/apache/incubator-systemml/issues/307/labels{/name}","comments_url":"https://api.github.com/repos/apache/incubator-systemml/issues/307/comments","events_url":"https://api.github.com/repos/apache/incubator-systemml/issues/307/events","html_url":"https://github.com/apache/incubator-systemml/pull/307","id":193651320,"number":307,"title":"[SYSTEMML-769] [WIP] Support for native BLAS in SystemML","user":{"login":"niketanpansare","id":11686380,"avatar_url":"https://avatars.githubusercontent.com/u/11686380?v=3","gravatar_id":"","url":"https://api.github.com/users/niketanpansare","html_url":"https://github.com/niketanpansare","followers_url":"https://api.github.com/users/niketanpansare/followers","following_url":"https://api.github.com/users/niketanpansare/following{/other_user}","gists_url":"https://api.github.com/users/niketanpansare/gists{/gist_id}","starred_url":"https://api.github.com/users/niketanpansare/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/niketanpansare/subscriptions","organizations_url":"https://api.github.com/users/niketanpansare/orgs","repos_url":"https://api.github.com/users/niketanpansare/repos","events_url":"https://api.github.com/users/niketanpansare/events{/privacy}","received_events_url":"https://api.github.com/users/niketanpansare/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":16,"created_at":"2016-12-06T00:34:14Z","updated_at":"2016-12-20T00:45:51Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/incubator-systemml/pulls/307","html_url":"https://github.com/apache/incubator-systemml/pull/307","diff_url":"https://github.com/apache/incubator-systemml/pull/307.diff","patch_url":"https://github.com/apache/incubator-systemml/pull/307.patch"},"body":"This is a standing PR to facilitate discussion on whether or not, we should support native BLAS in SystemML. After discussion and after resolving the issues with deployment, we can decide whether to turn on this feature by default. Since I wanted feedback from community before proceeding ahead, I did not complete the PR. The remaining tasks are:\r\n\r\n- Generalize to other BLAS, not just MKL. This would also involve completing the CMake file.\r\n- Add other operations: conv2d_backward_*, etc.\r\n\r\nI ran some preliminary performance experiments comparing conv2d with/without sparse+caching and with/without native BLAS. I provided fairly large memory budget (-Xmx20g -Xms20g -Xmn2048m -server) and used Open JDK 1.8 64-Bit Server VM. The script tested the performance of conv2d using four commonly used setups for 1000 iterations:\r\n\r\n```python\r\nmax_iterations = 1000\r\nsetup = $2\r\nnumFilters = -1\r\nnumChannels = -1\r\nfilterSize = -1\r\npad = -1\r\nif(setup == 1) {\r\n        numFilters = 20\r\n        numChannels = 1\r\n        filterSize = 5\r\n        pad = 0\r\n}\r\nelse if(setup == 2) {\r\n        numFilters = 50\r\n        numChannels = 20\r\n        filterSize = 5\r\n        pad = 0\r\n}\r\nelse if(setup == 3) {\r\n        numFilters = 20\r\n        numChannels = 1\r\n        filterSize = 3\r\n        pad = 1\r\n}\r\nelse if(setup == 4) {\r\n        numFilters = 50\r\n        numChannels = 20\r\n        filterSize = 3\r\n        pad = 1\r\n}\r\nelse {\r\n        stop('Incorrect setup (needs to be [1, 4]).')\r\n}\r\nimgSize = 28\r\nn = 60000\r\nX = rand(rows=n, cols=numChannels*imgSize*imgSize)\r\nbatch_size = 64\r\nw = rand(rows=numFilters, cols=numChannels*filterSize*filterSize)\r\nP = (imgSize + 2 * pad - filterSize)  + 1\r\nfoo = matrix(0, rows=n, cols=numFilters*P*P)\r\nfor(iter in 1:max_iterations) {\r\n        beg = (iter * batch_size) %% n + 1\r\n        end = min(n, beg + batch_size)\r\n        X_batch = X[beg:end, ]\r\n        n_batch = nrow(X_batch)\r\n        convOut_1 = conv2d(X_batch, w, input_shape=[n_batch,numChannels,imgSize,imgSize], filter_shape=[numFilters,numChannels,filterSize,filterSize], padding=[pad,pad], stride=[1,1])\r\n        foo = convOut_1\r\n}\r\nprint(sum(foo))\r\n```\r\n\r\nTo compile the native SystemML library, please use:\r\n```bash\r\nexport MKLROOT=/opt/intel/mkl\r\nexport JAVA_HOME=....\r\n# Please go to https://software.intel.com/en-us/articles/intel-mkl-link-line-advisor to find LINKER_OPTIONS and COMPILER_OPTIONS\r\nexport LINKER_OPTIONS=\" -L${MKLROOT}/lib/intel64 -Wl,--no-as-needed -lmkl_rt -lpthread -lm -ldl\"\r\nexport COMPILER_OPTIONS=\" -m64 -I${MKLROOT}/include\"\r\ng++ -shared -fPIC -o libsystemml.so systemml.cpp -I. -I$JAVA_HOME/include -I$JAVA_HOME/include/linux -lm -fopenmp -O3 $LINKER_OPTIONS $COMPILER_OPTIONS \r\n```\r\n\r\nPlease see below the results of the experiments. Both sparse and caching are disabled for the setup `SystemML_native` and `SystemML_CP`.\r\n\r\n```bash\r\n,Number of Iterations, Setup, Time in seconds\r\nSystemML_native,1000,1,7.103096398\r\nSystemML_CP,1000,1,6.498525426\r\nSystemML_CP_WithCacheNSparseEnabled,1000,1,7.195620854\r\nTensorflow,1000,1,4.071731716\r\nSystemML_native,1000,2,31.315343223\r\nSystemML_CP,1000,2,81.769984552\r\nSystemML_CP_WithCacheNSparseEnabled,1000,2,101.274622939\r\nTensorflow,1000,2,33.476548341\r\nSystemML_native,1000,3,7.662274848\r\nSystemML_CP,1000,3,6.355272119\r\nSystemML_CP_WithCacheNSparseEnabled,1000,3,7.607337158\r\nTensorflow,1000,3,3.837932081\r\nSystemML_native,1000,4,26.638438614\r\nSystemML_CP,1000,4,49.716594505\r\nSystemML_CP_WithCacheNSparseEnabled,1000,4,71.542244484\r\nTensorflow,1000,4,26.395180006\r\n```\r\n\r\nThere are some additional overhead cost (such as initial compilation/validation, reuse of previously allocated but non-zeroed array, dynamic recompilation, GC, etc) which we have not yet optimized. These cost are beyond the scope of this PR and some of them are inherent to our design principles. We can work on them in a separate PR :)\r\n\r\n@mboehm7 @bertholdreinwald @dusenberrymw @frreiss @prithvirajsen @fschueler @nakul02 @asurve   @deroneriksson   I understand the above experiments might not be sufficient to accept the change and would welcome your feedback on additional experiments/setups. I would also appreciate if some of you are willing to help me with these experiments too ;)\r\n\r\nHere are the shapes of the matrix multiplication for the four setups:\r\n```bash\r\nSetup 1:\r\n64 parallel matrix multiplication of shape (20, 25) %*% (25, 576) executed 1000 times.\r\n \r\nSetup 2:\r\n64 parallel matrix multiplication of shape (50, 500) %*% (500, 576) executed 1000 times.\r\n \r\nSetup 3:\r\n64 parallel matrix multiplication of shape (20, 25) %*% (25, 784) executed 1000 times.\r\n \r\nSetup 4:\r\n64 parallel matrix multiplication of shape (50, 500) %*% (500, 784) executed 1000 times.\r\n```\r\nI will provide an update soon comparing the results of the above matrix multiplications. If you are interested, here are the respective code path for the matrix multiplications:\r\n\r\n- CP: https://github.com/apache/incubator-systemml/blob/master/src/main/java/org/apache/sysml/runtime/matrix/data/LibMatrixDNN.java#L327\r\n\r\n- Native: https://github.com/niketanpansare/incubator-systemml/blob/for_cpp/src/main/cpp/systemml.cpp#L163"},"comment":{"url":"https://api.github.com/repos/apache/incubator-systemml/issues/comments/268121620","html_url":"https://github.com/apache/incubator-systemml/pull/307#issuecomment-268121620","issue_url":"https://api.github.com/repos/apache/incubator-systemml/issues/307","id":268121620,"user":{"login":"niketanpansare","id":11686380,"avatar_url":"https://avatars.githubusercontent.com/u/11686380?v=3","gravatar_id":"","url":"https://api.github.com/users/niketanpansare","html_url":"https://github.com/niketanpansare","followers_url":"https://api.github.com/users/niketanpansare/followers","following_url":"https://api.github.com/users/niketanpansare/following{/other_user}","gists_url":"https://api.github.com/users/niketanpansare/gists{/gist_id}","starred_url":"https://api.github.com/users/niketanpansare/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/niketanpansare/subscriptions","organizations_url":"https://api.github.com/users/niketanpansare/orgs","repos_url":"https://api.github.com/users/niketanpansare/repos","events_url":"https://api.github.com/users/niketanpansare/events{/privacy}","received_events_url":"https://api.github.com/users/niketanpansare/received_events","type":"User","site_admin":false},"created_at":"2016-12-20T00:45:51Z","updated_at":"2016-12-20T00:45:51Z","body":"To summarize, I tried following with openblas, but still saw performance degradation:\r\n\r\n1. Compiling OpenBLAS from source:\r\n\t```bash\r\n\tgit clone https://github.com/xianyi/OpenBLAS\r\n\tcd OpenBLAS\r\n\tmake FC=gfortran\r\n\tsudo make PREFIX=/opt/openblas install\r\n\t```\r\n\r\n2. Using released OpenBLAS:\r\n\t```bash\r\n\tsudo yum install openblas\r\n\tsudo ln -s /lib64/libopenblas.so.0 /usr/lib/libopenblas.so\r\n\t```\r\n\r\n3. Also, commented any threading logic in systemml.cpp to rule out possibility of bug due to extension APIs:\r\n\t```python\r\n        //if(NUM_THREADS == -1) {\r\n        //      NUM_THREADS = openblas_get_num_threads();\r\n        //}\r\n        // openblas_set_num_threads(NUM_THREADS);\r\n\t```\r\n\r\n4. Even though I was using newer Centos 7 and gcc 4.8 (both supporting avx instructions), I also tried updating binutils and reinstalling OpenBLAS as per https://github.com/xianyi/OpenBLAS/wiki/faq#binutils\r\n\r\nMy guess is that this is due to a performance bug in OpenBLAS on multi-socket machine: https://github.com/xianyi/OpenBLAS/issues/611\r\n\r\n```bash\r\n$ cat /proc/cpuinfo | grep \"physical id\" | sort -u | wc -l\r\n2\r\n```"}}
{"actor":{"display_login":"niketanpansare"}}