{"action":"created","issue":{"url":"https://api.github.com/repos/ispc/ispc/issues/1175","repository_url":"https://api.github.com/repos/ispc/ispc","labels_url":"https://api.github.com/repos/ispc/ispc/issues/1175/labels{/name}","comments_url":"https://api.github.com/repos/ispc/ispc/issues/1175/comments","events_url":"https://api.github.com/repos/ispc/ispc/issues/1175/events","html_url":"https://github.com/ispc/ispc/issues/1175","id":134984018,"number":1175,"title":"STREAM Benchmark Performance","user":{"login":"lcw","id":117908,"avatar_url":"https://avatars.githubusercontent.com/u/117908?v=3","gravatar_id":"","url":"https://api.github.com/users/lcw","html_url":"https://github.com/lcw","followers_url":"https://api.github.com/users/lcw/followers","following_url":"https://api.github.com/users/lcw/following{/other_user}","gists_url":"https://api.github.com/users/lcw/gists{/gist_id}","starred_url":"https://api.github.com/users/lcw/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lcw/subscriptions","organizations_url":"https://api.github.com/users/lcw/orgs","repos_url":"https://api.github.com/users/lcw/repos","events_url":"https://api.github.com/users/lcw/events{/privacy}","received_events_url":"https://api.github.com/users/lcw/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2016-02-19T21:49:43Z","updated_at":"2016-07-29T08:16:12Z","closed_at":null,"body":"I have been comparing the performance of `ispc` against `icc` for the kernels in the [STREAM benchmark](https://www.cs.virginia.edu/stream/).  Currently the `icc` compiled STREAM benchmark is about 1.3x faster than my implementation of the STEAM benchmark using `ispc` (which can be found [here](https://github.com/lcw/stream_ispc)).  If possible I would like to improve the `ispc` version so that it is on par with the `icc` version.\r\n\r\nIs it expected that `ispc` should be able to perform as well as `icc` compiled code?  Do you have any ideas on how I can improve the `ispc` version of the benchmark?\r\n\r\nThe code I used can be found at <https://github.com/lcw/stream_ispc> and below is a summary of my benchmark results.\r\n\r\nI compared `build/stream` and `build/stream_ispc` on a dual socket Xeon\r\nCPU E5-2698 v3 system. I used the following build flags\r\n\r\n    CC=icc\r\n    CFLAGS=-std=gnu99 -g -xHOST -O3 -ffreestanding -openmp\r\n\r\n    CXX=icpc\r\n    CXXFLAGS=-g -xHOST -O3 -ffreestanding -openmp -DISPC_USE_OMP\r\n\r\n    ISPC = ispc\r\n    ISPCFLAGS = --target=avx2-i32x8 --pic --opt=force-aligned-memory --werror\r\n\r\nHere is the version information for the compilers we are using:\r\n\r\n``` {.bash .rundoc-block rundoc-language=\"sh\" rundoc-exports=\"both\"}\r\nicc --version\r\n```\r\n\r\n``` {.example}\r\nicc (ICC) 16.0.1 20151021\r\nCopyright (C) 1985-2015 Intel Corporation.  All rights reserved.\r\n```\r\n\r\n``` {.bash .rundoc-block rundoc-language=\"sh\" rundoc-exports=\"both\"}\r\nispc --version\r\n```\r\n\r\n``` {.example}\r\nIntel(r) SPMD Program Compiler (ispc), 1.9.0 (build commit 89dfbf2125fc2cba @ 20160212, LLVM 3.8)\r\n```\r\n\r\nHere is the output from the STREAM benchmark compiled with `icc`:\r\n\r\n``` {.bash .rundoc-block rundoc-language=\"sh\" rundoc-exports=\"both\"}\r\nOMP_PLACES=cores OMP_DISPLAY_ENV=true ./build/stream\r\n```\r\n\r\n``` {.example}\r\n-------------------------------------------------------------\r\nSTREAM version $Revision: 5.10 $\r\n-------------------------------------------------------------\r\nThis system uses 4 bytes per array element.\r\n-------------------------------------------------------------\r\nArray size = 80000000 (elements), Offset = 0 (elements)\r\nMemory per array = 305.2 MiB (= 0.3 GiB).\r\nTotal memory required = 915.5 MiB (= 0.9 GiB).\r\nEach kernel will be executed 10 times.\r\n The *best* time for each kernel (excluding the first iteration)\r\n will be used to compute the reported bandwidth.\r\n-------------------------------------------------------------\r\n\r\nOPENMP DISPLAY ENVIRONMENT BEGIN\r\n   _OPENMP='201307'\r\n  [host] OMP_CANCELLATION='FALSE'\r\n  [host] OMP_DISPLAY_ENV='TRUE'\r\n  [host] OMP_DYNAMIC='FALSE'\r\n  [host] OMP_MAX_ACTIVE_LEVELS='2147483647'\r\n  [host] OMP_NESTED='FALSE'\r\n  [host] OMP_NUM_THREADS: value is not defined\r\n  [host] OMP_PLACES='cores'\r\n  [host] OMP_PROC_BIND='spread'\r\n  [host] OMP_SCHEDULE='static'\r\n  [host] OMP_STACKSIZE='4M'\r\n  [host] OMP_THREAD_LIMIT='2147483647'\r\n  [host] OMP_WAIT_POLICY='PASSIVE'\r\nOPENMP DISPLAY ENVIRONMENT END\r\n\r\n\r\nNumber of Threads requested = 32\r\nNumber of Threads counted = 32\r\n-------------------------------------------------------------\r\nYour clock granularity/precision appears to be 1 microseconds.\r\nEach test below will take on the order of 5419 microseconds.\r\n   (= 5419 clock ticks)\r\nIncrease the size of the arrays if this shows that\r\nyou are not getting at least 20 clock ticks per test.\r\n-------------------------------------------------------------\r\nWARNING -- The above is only a rough guideline.\r\nFor best results, please be sure you know the\r\nprecision of your system timer.\r\n-------------------------------------------------------------\r\nFunction    Best Rate MB/s  Avg time     Min time     Max time\r\nCopy:          104902.7     0.006141     0.006101     0.006308\r\nScale:         106522.0     0.006039     0.006008     0.006146\r\nAdd:           112215.9     0.008605     0.008555     0.008762\r\nTriad:         112097.2     0.008595     0.008564     0.008710\r\n-------------------------------------------------------------\r\nSolution Validates: avg error less than 1.000000e-06 on all three arrays\r\nResults Validation Verbose Results:\r\n    Expected a(1), b(1), c(1): 1153300692992.000000 230660145152.000000 307546849280.000000\r\n    Observed a(1), b(1), c(1): 1153300824064.000000 230660161536.000000 307546882048.000000\r\n    Rel Errors on a, b, c:     1.136495e-07 7.103091e-08 1.065464e-07\r\n-------------------------------------------------------------\r\n```\r\n\r\nHere is the output from the modified STREAM benchmark with kernels\r\ncompiled with `ispc`:\r\n\r\n``` {.bash .rundoc-block rundoc-language=\"sh\" rundoc-exports=\"both\"}\r\nOMP_PLACES=cores OMP_DISPLAY_ENV=true ./build/stream_ispc\r\n```\r\n\r\n``` {.example}\r\nArray size = 80000000 (elements)\r\nMemory per array = 305.2 MiB (= 0.3 GiB).\r\nTotal memory required = 915.5 MiB (= 0.9 GiB).\r\nChunk size: 16384\r\nPage size: 4096\r\nCache line size: 64\r\nsizeof(STREAM_TYPE): 4\r\nEach kernel will be executed 10 times.\r\n The *best* time for each kernel (excluding the first iteration)\r\n will be used to compute the reported bandwidth.\r\n-------------------------------------------------------------\r\n\r\nOPENMP DISPLAY ENVIRONMENT BEGIN\r\n   _OPENMP='201307'\r\n  [host] OMP_CANCELLATION='FALSE'\r\n  [host] OMP_DISPLAY_ENV='TRUE'\r\n  [host] OMP_DYNAMIC='FALSE'\r\n  [host] OMP_MAX_ACTIVE_LEVELS='2147483647'\r\n  [host] OMP_NESTED='FALSE'\r\n  [host] OMP_NUM_THREADS: value is not defined\r\n  [host] OMP_PLACES='cores'\r\n  [host] OMP_PROC_BIND='spread'\r\n  [host] OMP_SCHEDULE='static'\r\n  [host] OMP_STACKSIZE='4M'\r\n  [host] OMP_THREAD_LIMIT='2147483647'\r\n  [host] OMP_WAIT_POLICY='PASSIVE'\r\nOPENMP DISPLAY ENVIRONMENT END\r\n\r\n\r\n-------------------------------------------------------------\r\nEach test below will take on the order of 6482 microseconds.\r\n-------------------------------------------------------------\r\n-------------------------------------------------------------\r\nFunction    Best Rate MB/s  Avg time     Min time     Max time\r\nCopy:           75179.7     0.008546     0.008513     0.008603\r\nScale:          73558.4     0.008729     0.008701     0.008792\r\nAdd:            83152.5     0.011573     0.011545     0.011613\r\nTriad:          83805.1     0.011485     0.011455     0.011520\r\n-------------------------------------------------------------\r\nSolution Validates: avg error less than 1.000000e-06 on all three arrays\r\nResults Validation Verbose Results:\r\n    Expected a(1), b(1), c(1): 1153300692992.000000 230660145152.000000 307546849280.000000\r\n    Observed a(1), b(1), c(1): 1153300824064.000000 230660161536.000000 307546882048.000000\r\n    Rel Errors on a, b, c:     1.136495e-07 7.103091e-08 1.065464e-07\r\n-------------------------------------------------------------\r\n```\r\n\r\nAs we can see the `icc` version is about 1.3x faster than the `ispc`\r\nversion. I tried various memory alignment procedures but could not\r\nimprove the performance of the `ispc` version of the benchmark.\r\n\r\ncc: @inducer, @tcew, @jkozdon"},"comment":{"url":"https://api.github.com/repos/ispc/ispc/issues/comments/236121717","html_url":"https://github.com/ispc/ispc/issues/1175#issuecomment-236121717","issue_url":"https://api.github.com/repos/ispc/ispc/issues/1175","id":236121717,"user":{"login":"dbabokin","id":3514138,"avatar_url":"https://avatars.githubusercontent.com/u/3514138?v=3","gravatar_id":"","url":"https://api.github.com/users/dbabokin","html_url":"https://github.com/dbabokin","followers_url":"https://api.github.com/users/dbabokin/followers","following_url":"https://api.github.com/users/dbabokin/following{/other_user}","gists_url":"https://api.github.com/users/dbabokin/gists{/gist_id}","starred_url":"https://api.github.com/users/dbabokin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dbabokin/subscriptions","organizations_url":"https://api.github.com/users/dbabokin/orgs","repos_url":"https://api.github.com/users/dbabokin/repos","events_url":"https://api.github.com/users/dbabokin/events{/privacy}","received_events_url":"https://api.github.com/users/dbabokin/received_events","type":"User","site_admin":false},"created_at":"2016-07-29T08:16:12Z","updated_at":"2016-07-29T08:16:12Z","body":"@lcw @inducer \r\n#1181 is not complete and not tested well. I'll ask Andrey @Shishpan to get back to this set of changes and check what needs to be done to complete it."}}
{"actor":{"display_login":"dbabokin"}}