{"action":"created","issue":{"url":"https://api.github.com/repos/cms-patatrack/cmssw/issues/125","repository_url":"https://api.github.com/repos/cms-patatrack/cmssw","labels_url":"https://api.github.com/repos/cms-patatrack/cmssw/issues/125/labels{/name}","comments_url":"https://api.github.com/repos/cms-patatrack/cmssw/issues/125/comments","events_url":"https://api.github.com/repos/cms-patatrack/cmssw/issues/125/events","html_url":"https://github.com/cms-patatrack/cmssw/pull/125","id":349094704,"node_id":"MDExOlB1bGxSZXF1ZXN0MjA3MjkwMzg2","number":125,"title":"Speed up CPU side of GPU rechits","user":{"login":"makortel","id":33993,"node_id":"MDQ6VXNlcjMzOTkz","avatar_url":"https://avatars3.githubusercontent.com/u/33993?v=4","gravatar_id":"","url":"https://api.github.com/users/makortel","html_url":"https://github.com/makortel","followers_url":"https://api.github.com/users/makortel/followers","following_url":"https://api.github.com/users/makortel/following{/other_user}","gists_url":"https://api.github.com/users/makortel/gists{/gist_id}","starred_url":"https://api.github.com/users/makortel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/makortel/subscriptions","organizations_url":"https://api.github.com/users/makortel/orgs","repos_url":"https://api.github.com/users/makortel/repos","events_url":"https://api.github.com/users/makortel/events{/privacy}","received_events_url":"https://api.github.com/users/makortel/received_events","type":"User","site_admin":false},"labels":[{"id":771304797,"node_id":"MDU6TGFiZWw3NzEzMDQ3OTc=","url":"https://api.github.com/repos/cms-patatrack/cmssw/labels/enhancement","name":"enhancement","color":"84b6eb","default":true}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":17,"created_at":"2018-08-09T11:52:31Z","updated_at":"2018-08-13T09:52:23Z","closed_at":null,"author_association":"NONE","pull_request":{"url":"https://api.github.com/repos/cms-patatrack/cmssw/pulls/125","html_url":"https://github.com/cms-patatrack/cmssw/pull/125","diff_url":"https://github.com/cms-patatrack/cmssw/pull/125.diff","patch_url":"https://github.com/cms-patatrack/cmssw/pull/125.patch"},"body":"`HitsOnCPU` allocates the (pinned) host memory on each event. Profiler shows that this is rather costly (more than half of the wall clock time spent in the kernels...). The first commit moves the allocations as the first action in the function, because I first saw a peculiar profile where one of the `cudaMallocHost` took very long time and delayed the queueing of the asynchronous memory copies, thus making the `acquire()` to occupy CPU.\r\n\r\nOf course this didn't significantly improve the situation though, so the second commit changes the pattern to the same as in raw2cluster, i.e. allocate (also host side) buffers once per module (i.e. per job per EDM stream), and the event product just holds pointers to them. Not ideal, but faster (in a single event the wall clock time of `acquire()` reduces from ~240 us to ~170 us).\r\n\r\nWe can of course discuss whether this approach is what we really want (probably not), but at the moment it seems to be the fastest.\r\n\r\n@fwyzard @VinInn @felicepantaleo "},"comment":{"url":"https://api.github.com/repos/cms-patatrack/cmssw/issues/comments/412466337","html_url":"https://github.com/cms-patatrack/cmssw/pull/125#issuecomment-412466337","issue_url":"https://api.github.com/repos/cms-patatrack/cmssw/issues/125","id":412466337,"node_id":"MDEyOklzc3VlQ29tbWVudDQxMjQ2NjMzNw==","user":{"login":"makortel","id":33993,"node_id":"MDQ6VXNlcjMzOTkz","avatar_url":"https://avatars3.githubusercontent.com/u/33993?v=4","gravatar_id":"","url":"https://api.github.com/users/makortel","html_url":"https://github.com/makortel","followers_url":"https://api.github.com/users/makortel/followers","following_url":"https://api.github.com/users/makortel/following{/other_user}","gists_url":"https://api.github.com/users/makortel/gists{/gist_id}","starred_url":"https://api.github.com/users/makortel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/makortel/subscriptions","organizations_url":"https://api.github.com/users/makortel/orgs","repos_url":"https://api.github.com/users/makortel/repos","events_url":"https://api.github.com/users/makortel/events{/privacy}","received_events_url":"https://api.github.com/users/makortel/received_events","type":"User","site_admin":false},"created_at":"2018-08-13T09:52:23Z","updated_at":"2018-08-13T09:52:23Z","author_association":"NONE","body":"Since the `initcheck` (presumably) started to complain after `cudaMallocPitch`+`cudaMemcpy2DAsync` I suppose it flags the padding. I don't know why anything whould be accessing that though. Maybe `cudaMemcpy2DAsync`? But even with that I'd find the access a bit strange as the target host memory does not have any padding."}}
{"actor":{"display_login":"makortel"}}