{"action":"created","issue":{"url":"https://api.github.com/repos/trilinos/Trilinos/issues/2518","repository_url":"https://api.github.com/repos/trilinos/Trilinos","labels_url":"https://api.github.com/repos/trilinos/Trilinos/issues/2518/labels{/name}","comments_url":"https://api.github.com/repos/trilinos/Trilinos/issues/2518/comments","events_url":"https://api.github.com/repos/trilinos/Trilinos/issues/2518/events","html_url":"https://github.com/trilinos/Trilinos/issues/2518","id":312626709,"number":2518,"title":"Question: Tpetra: Appropriate data structure for block data","user":{"login":"ctanis","id":405078,"avatar_url":"https://avatars0.githubusercontent.com/u/405078?v=4","gravatar_id":"","url":"https://api.github.com/users/ctanis","html_url":"https://github.com/ctanis","followers_url":"https://api.github.com/users/ctanis/followers","following_url":"https://api.github.com/users/ctanis/following{/other_user}","gists_url":"https://api.github.com/users/ctanis/gists{/gist_id}","starred_url":"https://api.github.com/users/ctanis/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ctanis/subscriptions","organizations_url":"https://api.github.com/users/ctanis/orgs","repos_url":"https://api.github.com/users/ctanis/repos","events_url":"https://api.github.com/users/ctanis/events{/privacy}","received_events_url":"https://api.github.com/users/ctanis/received_events","type":"User","site_admin":false},"labels":[{"id":289606028,"url":"https://api.github.com/repos/trilinos/Trilinos/labels/Tpetra","name":"Tpetra","color":"5319e7","default":false},{"id":287520108,"url":"https://api.github.com/repos/trilinos/Trilinos/labels/question","name":"question","color":"cc317c","default":true}],"state":"open","locked":false,"assignee":{"login":"kyungjoo-kim","id":6992602,"avatar_url":"https://avatars2.githubusercontent.com/u/6992602?v=4","gravatar_id":"","url":"https://api.github.com/users/kyungjoo-kim","html_url":"https://github.com/kyungjoo-kim","followers_url":"https://api.github.com/users/kyungjoo-kim/followers","following_url":"https://api.github.com/users/kyungjoo-kim/following{/other_user}","gists_url":"https://api.github.com/users/kyungjoo-kim/gists{/gist_id}","starred_url":"https://api.github.com/users/kyungjoo-kim/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kyungjoo-kim/subscriptions","organizations_url":"https://api.github.com/users/kyungjoo-kim/orgs","repos_url":"https://api.github.com/users/kyungjoo-kim/repos","events_url":"https://api.github.com/users/kyungjoo-kim/events{/privacy}","received_events_url":"https://api.github.com/users/kyungjoo-kim/received_events","type":"User","site_admin":false},"assignees":[{"login":"kyungjoo-kim","id":6992602,"avatar_url":"https://avatars2.githubusercontent.com/u/6992602?v=4","gravatar_id":"","url":"https://api.github.com/users/kyungjoo-kim","html_url":"https://github.com/kyungjoo-kim","followers_url":"https://api.github.com/users/kyungjoo-kim/followers","following_url":"https://api.github.com/users/kyungjoo-kim/following{/other_user}","gists_url":"https://api.github.com/users/kyungjoo-kim/gists{/gist_id}","starred_url":"https://api.github.com/users/kyungjoo-kim/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kyungjoo-kim/subscriptions","organizations_url":"https://api.github.com/users/kyungjoo-kim/orgs","repos_url":"https://api.github.com/users/kyungjoo-kim/repos","events_url":"https://api.github.com/users/kyungjoo-kim/events{/privacy}","received_events_url":"https://api.github.com/users/kyungjoo-kim/received_events","type":"User","site_admin":false},{"login":"ndellingwood","id":14129110,"avatar_url":"https://avatars3.githubusercontent.com/u/14129110?v=4","gravatar_id":"","url":"https://api.github.com/users/ndellingwood","html_url":"https://github.com/ndellingwood","followers_url":"https://api.github.com/users/ndellingwood/followers","following_url":"https://api.github.com/users/ndellingwood/following{/other_user}","gists_url":"https://api.github.com/users/ndellingwood/gists{/gist_id}","starred_url":"https://api.github.com/users/ndellingwood/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ndellingwood/subscriptions","organizations_url":"https://api.github.com/users/ndellingwood/orgs","repos_url":"https://api.github.com/users/ndellingwood/repos","events_url":"https://api.github.com/users/ndellingwood/events{/privacy}","received_events_url":"https://api.github.com/users/ndellingwood/received_events","type":"User","site_admin":false}],"milestone":null,"comments":49,"created_at":"2018-04-09T17:54:27Z","updated_at":"2018-05-24T17:30:08Z","closed_at":null,"author_association":"NONE","body":"I am getting terrible performance with Experimental::BlockCrsMatrix  and MultiVec (BlockCrsVector doesn't compile).   What is currently the best way to deal with multiple degrees of freedom per mesh point?\r\n\r\n<!---\r\nReplace <teamName> below with the appropriate Trilinos package/team name.\r\n-->\r\n@trilinos/tpetra\r\n\r\n<!---\r\nLables:  Choose any applicable package names from the Labels drop-down on the\r\nright.  Additionally, choose a label to indicate the type of issue, for\r\ninstance, bug, build, documentation, enhancement, etc.\r\n-->\r\n\r\n## Expectations\r\n<!---\r\nTell us what you think should happen, how you think things should work, what\r\nyou would like to see in the documentation, etc.\r\n-->\r\n\r\nExperimental::BlockCrsMatrix.apply should be faster in parallel, or I should be doing this in a different way.\r\n\r\n## Current Behavior\r\n<!---\r\nTell us how the current behavior fails to meet your expectations in some way.\r\n-->\r\n\r\nUsing MPI (np = 2), BlockCrsMatrix.apply is taking approximately 100 times longer than in serial (still getting the correct answer, I think).\r\n\r\nThis is applying a block matrix with 1 degree of freedom to a MultiVector (not block) with 1 vector.\r\n\r\nAttempting to compile a simple example with BlockVector does not compile. \r\n[compilation_error.txt](https://github.com/trilinos/Trilinos/files/1891118/compilation_error.txt)\r\n\r\n\r\n\r\n## Motivation and Context\r\n<!---\r\nHow has this expectation failure affected you?  What are you trying to\r\naccomplish?  Why do we need to address this?  What does it have to do with\r\nanything?  Providing context helps us come up with a solution that is most\r\nuseful in the real world.\r\n-->\r\n\r\nI understand these are \"experimental\" data structures, so I would love to know the most appropriate way to go forward.\r\n\r\nAlso note:  I was using 1 degree of freedom per mesh point to make sure I was solving the problem correctly.  My imminent need is to have 4 or more per mesh point.  Potentially many more!\r\n\r\nI'm going to attempt to add a 'question' label after submitting this.  My github interface is somehow missing the label and assignee dropdowns."},"comment":{"url":"https://api.github.com/repos/trilinos/Trilinos/issues/comments/391796370","html_url":"https://github.com/trilinos/Trilinos/issues/2518#issuecomment-391796370","issue_url":"https://api.github.com/repos/trilinos/Trilinos/issues/2518","id":391796370,"user":{"login":"ctanis","id":405078,"avatar_url":"https://avatars0.githubusercontent.com/u/405078?v=4","gravatar_id":"","url":"https://api.github.com/users/ctanis","html_url":"https://github.com/ctanis","followers_url":"https://api.github.com/users/ctanis/followers","following_url":"https://api.github.com/users/ctanis/following{/other_user}","gists_url":"https://api.github.com/users/ctanis/gists{/gist_id}","starred_url":"https://api.github.com/users/ctanis/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ctanis/subscriptions","organizations_url":"https://api.github.com/users/ctanis/orgs","repos_url":"https://api.github.com/users/ctanis/repos","events_url":"https://api.github.com/users/ctanis/events{/privacy}","received_events_url":"https://api.github.com/users/ctanis/received_events","type":"User","site_admin":false},"created_at":"2018-05-24T17:30:08Z","updated_at":"2018-05-24T17:30:08Z","author_association":"NONE","body":"@mhoemmen \r\n\r\n> How do you build the graph's row Map?\r\n\r\nI'm glad you asked this, as I've been revisiting this part of my code today.  I am  hoping that part of my problem is that I've misunderstood something and used a naive column map in my graph as well.\r\nThe row map is a contiguous map based on the range of global ids owned by each rank.  I used a column map to map local indices to phantom nodes owned by other processes, but I think this is causing a lot more communication than necessary during the matrix-vector multiply.  \r\n\r\nLogically I have everything that I need on each MPI rank, but the matrix needs to know about vector data on other elements.  How should this be done?  I couldn't find a good example of this with an unstructured problem.\r\n\r\n> CudaUVMSpace allocations live on the GPU, but are accessible from host.\r\nHow does this work with GPU page faults where UVM can be larger than GPU memory?  Your point on simultaneous access is well-taken!\r\n\r\nThanks for all your help."}}
{"actor":{"display_login":"ctanis"}}