{"action":"created","issue":{"url":"https://api.github.com/repos/kokkos/kokkos/issues/1891","repository_url":"https://api.github.com/repos/kokkos/kokkos","labels_url":"https://api.github.com/repos/kokkos/kokkos/issues/1891/labels{/name}","comments_url":"https://api.github.com/repos/kokkos/kokkos/issues/1891/comments","events_url":"https://api.github.com/repos/kokkos/kokkos/issues/1891/events","html_url":"https://github.com/kokkos/kokkos/issues/1891","id":378968423,"node_id":"MDU6SXNzdWUzNzg5Njg0MjM=","number":1891,"title":"Default-layout shared memory view is slower than Kokkos::LayoutRight on CUDA backend","user":{"login":"Char-Aznable","id":32589713,"node_id":"MDQ6VXNlcjMyNTg5NzEz","avatar_url":"https://avatars1.githubusercontent.com/u/32589713?v=4","gravatar_id":"","url":"https://api.github.com/users/Char-Aznable","html_url":"https://github.com/Char-Aznable","followers_url":"https://api.github.com/users/Char-Aznable/followers","following_url":"https://api.github.com/users/Char-Aznable/following{/other_user}","gists_url":"https://api.github.com/users/Char-Aznable/gists{/gist_id}","starred_url":"https://api.github.com/users/Char-Aznable/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Char-Aznable/subscriptions","organizations_url":"https://api.github.com/users/Char-Aznable/orgs","repos_url":"https://api.github.com/users/Char-Aznable/repos","events_url":"https://api.github.com/users/Char-Aznable/events{/privacy}","received_events_url":"https://api.github.com/users/Char-Aznable/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2018-11-09T00:00:37Z","updated_at":"2018-11-10T19:10:24Z","closed_at":null,"author_association":"NONE","body":"I found using a `Kokkos::LayoutRight` for the shared-memory view on CUDA backend boosts the performance by about 34% than using the default layout (which is `Kokkos::LayoutLeft` on CUDA) in the following setting:\r\n\r\nI have a view of 2^25 double that is divided into 2^17 team of 2^3 threads x 2^5 vector lanes. I'm copying the elements from global to shared memory in each team, do a `ThreadVectorRange` scan for each thread in each team and copy the results back to global memory.  \r\n\r\nDoes this means a potential bank conflict in the default left-layout view? Should one use `Kokkos::LayoutRight` for shared memory view when it's parallelly accessed by ThreadVector in the second dimension or am I missing something? \r\n\r\nHere's the code and results:\r\n```c++\r\n#include<iostream>\r\n#include <utility>\r\n#include <type_traits>\r\n#include <Kokkos_Core.hpp>\r\n#include <Kokkos_DualView.hpp>\r\nusing ExecutionSpace = Kokkos::DefaultExecutionSpace;\r\nusing namespace std;\r\n\r\nusing TeamPolicy = Kokkos::TeamPolicy<ExecutionSpace>;\r\nusing Team = TeamPolicy::member_type;\r\nusing SharedSpace = ExecutionSpace::scratch_memory_space;\r\nusing UnmanagedTraits = Kokkos::MemoryTraits<Kokkos::Unmanaged>;\r\nstatic constexpr std::size_t shmemAlignBytes = 8;\r\nstatic constexpr std::size_t n = 1ul << 25;\r\nstatic constexpr std::size_t teamSize = 1ul << 3;\r\nstatic constexpr std::size_t nVectors = 1ul << 5;\r\nstatic constexpr std::size_t nTeams = n / (teamSize * nVectors);\r\n\r\ntemplate < class T, class L = void >\r\nusing SharedView = std::conditional_t<std::is_same<L,void>::value, \r\n  Kokkos::View<T,SharedSpace,UnmanagedTraits>,\r\n  Kokkos::View<T,L,SharedSpace,UnmanagedTraits>>;\r\n\r\n//Wrapper class for doing all the work in shared memory\r\ntemplate < class L=void >\r\nstruct ShMemory {\r\n  //If L is void (as by default), will use the default layout\r\n  SharedView<double[teamSize][nVectors],L> shView_;\r\n\r\n  std::size_t teamSize_, nVectors_;\r\n  KOKKOS_INLINE_FUNCTION\r\n  ShMemory(const Team& team) \r\n    :shView_(team.team_scratch(0)) \r\n    ,teamSize_(teamSize)\r\n    ,nVectors_(nVectors)\r\n  {\r\n  }\r\n\r\n  //copy from global to shared memory\r\n  KOKKOS_INLINE_FUNCTION\r\n  void copyFrom(const Team& team, const Kokkos::View<double[n],ExecutionSpace>& v) const {\r\n    const auto iTeam = team.league_rank();\r\n    Kokkos::parallel_for(Kokkos::TeamThreadRange(team,teamSize_),[&](const auto& i) {\r\n      Kokkos::parallel_for(Kokkos::ThreadVectorRange(team,nVectors_),[&](const auto& j) {\r\n        const auto iGlobal = iTeam*shView_.span()+i*nVectors_+j;\r\n        shView_(i,j) = v(iGlobal);\r\n      });\r\n    });\r\n    team.team_barrier();\r\n  }\r\n\r\n  //do a per-team scan using all vector lanes in each team\r\n  KOKKOS_INLINE_FUNCTION\r\n  void intraTeamScan(const Team& team) const {\r\n    Kokkos::parallel_for(Kokkos::TeamThreadRange(team,teamSize_),[&](const auto& i) {\r\n      Kokkos::parallel_scan(Kokkos::ThreadVectorRange(team,nVectors_),[&](const std::size_t& j, double& update, const bool& bFinal) {\r\n        update += shView_(i,j);\r\n        if(bFinal) { shView_(i,j) = update; }\r\n      });\r\n    });\r\n    team.team_barrier();\r\n  }\r\n\r\n  //copy from shared memory to global memory\r\n  KOKKOS_INLINE_FUNCTION\r\n  void copyTo(const Team& team, const Kokkos::View<double[n],ExecutionSpace>& v) const {\r\n    const auto iTeam = team.league_rank();\r\n    Kokkos::parallel_for(Kokkos::TeamThreadRange(team,teamSize_),[&](const auto& i) {\r\n      Kokkos::parallel_for(Kokkos::ThreadVectorRange(team,nVectors_),[&](const auto& j) {\r\n        const auto iGlobal = iTeam*shView_.span()+i*nVectors_+j;\r\n        v(iGlobal) = shView_(i,j);\r\n      });\r\n    });\r\n    team.team_barrier();\r\n  }\r\n};\r\n\r\n//reset input for next test case\r\ntemplate < class View >\r\ninline void reset(const View& v) {\r\n  Kokkos::parallel_for(v.extent(0),KOKKOS_LAMBDA(const auto& i) {\r\n    v(i) = 1;\r\n  });\r\n  Kokkos::fence();\r\n}\r\n\r\n//perform the test \r\ntemplate < class L, class Policy, class View >\r\ninline double doWork(Policy&& policy, const View& v) {\r\n  reset(v); \r\n  Kokkos::Timer timer;\r\n  Kokkos::parallel_for(std::forward<Policy>(policy), KOKKOS_LAMBDA(const typename std::decay_t<Policy>::member_type& team) {\r\n    ShMemory<L> shMemory(team);\r\n    shMemory.copyFrom(team,v);\r\n    shMemory.intraTeamScan(team);\r\n    shMemory.copyTo(team,v);\r\n  });\r\n  Kokkos::fence();\r\n  return timer.seconds();\r\n}\r\n\r\n//check if the results are correct\r\ntemplate < class View >\r\ninline void check(const View& v, const View& expect) {\r\n  if(v.extent(0) != expect.extent(0)) {\r\n    cout << \" mismatch sizes: \" << v.extent(0) << \"!=\" <<  expect.extent(0)\r\n      << endl;\r\n    exit(-1);\r\n  }\r\n  for(std::size_t i = 0; i < v.extent(0); ++i) {\r\n    if(v(i) != expect(i)) {\r\n      cout << \" wrong results at \" << i << \": \"\r\n        << v(i) << \"!=\" << expect(i) <<endl;\r\n      exit(-1);\r\n    }\r\n  }\r\n  cout << \" correct results\\n\";\r\n}\r\n\r\nint main(int argc, char* argv[]) {\r\n  Kokkos::initialize(argc,argv); {\r\n  using T = double;\r\n  Kokkos::DualView<T[n],ExecutionSpace> v(\"v\");\r\n  decltype(v)::t_host leftResults(\"leftResults\");\r\n  decltype(v)::t_host rightResults(\"rightResults\");\r\n  decltype(v)::t_host defaultResults(\"defaultResults\");\r\n  decltype(v)::t_host expectResults(\"expectResults\");\r\n\r\n  for(std::size_t i = 0; i < n; ++i) {\r\n    expectResults(i) = i%32 + 1;\r\n  }\r\n\r\n  auto policy = TeamPolicy(nTeams,teamSize,nVectors).set_scratch_size(0,Kokkos::PerTeam(shmemAlignBytes*teamSize*nVectors));\r\n  printf(\"#nItems = %10lu nTeams = %10lu teamSize = %10lu nVectors = %10lu\\n\",n,nTeams,teamSize,nVectors);\r\n\r\n  const auto dtDefault = doWork<void>(policy,v.d_view);\r\n  Kokkos::deep_copy(defaultResults,v.d_view);\r\n  const auto dtLeft = doWork<Kokkos::LayoutLeft>(policy,v.d_view);\r\n  Kokkos::deep_copy(leftResults,v.d_view);\r\n  const auto dtRight = doWork<Kokkos::LayoutRight>(policy,v.d_view);\r\n  Kokkos::deep_copy(rightResults,v.d_view);\r\n\r\n  cout << \"#Default layout got the \";\r\n  check(defaultResults,expectResults);\r\n  cout << \"#Left layout got the \";\r\n  check(leftResults,expectResults);\r\n  cout << \"#Right layout got the \";\r\n  check(rightResults,expectResults);\r\n\r\n  cout << \"#Time for each case in seconds:\\n\";\r\n  printf(\"#%30s%30s%30s\\n\",\"Default\",\"Right\",\"Left\");\r\n  printf(\"%30.15e%30.15e%30.15e\\n\",dtDefault,dtLeft,dtRight);\r\n\r\n  } Kokkos::finalize();\r\n}\r\n```\r\nOutput:\r\n```\r\n#nItems =   33554432 nTeams =     131072 teamSize =          8 nVectors =         32\r\n#Default layout got the  correct results\r\n#Left layout got the  correct results\r\n#Right layout got the  correct results\r\n#Time for each case in seconds:\r\n#                       Default                         Right                          Left\r\n         5.710971000000000e-03         5.695094000000000e-03         3.748197000000000e-03\r\n```"},"comment":{"url":"https://api.github.com/repos/kokkos/kokkos/issues/comments/437612309","html_url":"https://github.com/kokkos/kokkos/issues/1891#issuecomment-437612309","issue_url":"https://api.github.com/repos/kokkos/kokkos/issues/1891","id":437612309,"node_id":"MDEyOklzc3VlQ29tbWVudDQzNzYxMjMwOQ==","user":{"login":"Char-Aznable","id":32589713,"node_id":"MDQ6VXNlcjMyNTg5NzEz","avatar_url":"https://avatars1.githubusercontent.com/u/32589713?v=4","gravatar_id":"","url":"https://api.github.com/users/Char-Aznable","html_url":"https://github.com/Char-Aznable","followers_url":"https://api.github.com/users/Char-Aznable/followers","following_url":"https://api.github.com/users/Char-Aznable/following{/other_user}","gists_url":"https://api.github.com/users/Char-Aznable/gists{/gist_id}","starred_url":"https://api.github.com/users/Char-Aznable/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Char-Aznable/subscriptions","organizations_url":"https://api.github.com/users/Char-Aznable/orgs","repos_url":"https://api.github.com/users/Char-Aznable/repos","events_url":"https://api.github.com/users/Char-Aznable/events{/privacy}","received_events_url":"https://api.github.com/users/Char-Aznable/received_events","type":"User","site_admin":false},"created_at":"2018-11-10T19:10:24Z","updated_at":"2018-11-10T19:10:24Z","author_association":"NONE","body":"OK. I thought coalescing is not a big deal in shared memory as compared to global memory."}}
{"actor":{"display_login":"Char-Aznable"}}