{"action":"created","issue":{"url":"https://api.github.com/repos/inducer/pycuda/issues/171","repository_url":"https://api.github.com/repos/inducer/pycuda","labels_url":"https://api.github.com/repos/inducer/pycuda/issues/171/labels{/name}","comments_url":"https://api.github.com/repos/inducer/pycuda/issues/171/comments","events_url":"https://api.github.com/repos/inducer/pycuda/issues/171/events","html_url":"https://github.com/inducer/pycuda/pull/171","id":299061957,"number":171,"title":"Another attempt at supporting non-contiguous arrays","user":{"login":"SyamGadde","id":1247460,"avatar_url":"https://avatars1.githubusercontent.com/u/1247460?v=4","gravatar_id":"","url":"https://api.github.com/users/SyamGadde","html_url":"https://github.com/SyamGadde","followers_url":"https://api.github.com/users/SyamGadde/followers","following_url":"https://api.github.com/users/SyamGadde/following{/other_user}","gists_url":"https://api.github.com/users/SyamGadde/gists{/gist_id}","starred_url":"https://api.github.com/users/SyamGadde/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SyamGadde/subscriptions","organizations_url":"https://api.github.com/users/SyamGadde/orgs","repos_url":"https://api.github.com/users/SyamGadde/repos","events_url":"https://api.github.com/users/SyamGadde/events{/privacy}","received_events_url":"https://api.github.com/users/SyamGadde/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-02-21T17:35:47Z","updated_at":"2018-02-21T18:50:07Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/inducer/pycuda/pulls/171","html_url":"https://github.com/inducer/pycuda/pull/171","diff_url":"https://github.com/inducer/pycuda/pull/171.diff","patch_url":"https://github.com/inducer/pycuda/pull/171.patch"},"body":"Inspired by:\r\n  https://lists.tiker.net/pipermail/pycuda/2016-December/004986.html\r\n  https://gitlab.tiker.net/inducer/pycuda/merge_requests/1\r\n\r\nI tried a new approach to supporting non-contiguous arrays in PyCUDA (could be ported to PyOpenCL somewhat easily I think).  The goals (some elicited by the above discussion and comments in the WIP) were:\r\n- element-wise support for arbitrarily-strided arrays (including negative strides)\r\n- backwards compatibility with current uses of ``get_elwise_module`` and ``get_elwise_range_module``\r\n- limited performance impact on contiguous arrays\r\n- avoid use of '*', '%', and '/' in calculating indices, at least within the element-wise loop\r\n\r\nThe only way I could think to support all those goals was to delay compilation (and source generation) to call-time, to take advantage of knowledge of input array strides.  Contiguous arrays get the kernels that PyCUDA has always given them, non-contiguous arrays get specialized kernels.  The nice thing about doing this is that the actual shape and strides can be sent as '#define's to aid compiler optimization (could even help with the contiguous kernels).  The tricky thing about doing this is that some functions in the current implementation require the Module/Function before call-time, to get texrefs etc.  So I basically implemented a Proxy class for SourceModule, called ``DeferredSourceModule`` which also defers the generation of the values created by ``get_function()``, ``get_texref()``, etc. until call-time.\r\n\r\nTo make this all work, indexing (for non-contiguous arrays at least) for an array ``A`` needs to be ``A[A_i]``, rather than ``A[i]``.  If it detects matching contiguous arrays as inputs, then ``X_i`` is '#define'd to be ``i``, so kernels using the old method will still work (as long as the input arrays are contiguous and match in strides).  No regexes needed to transform the user-specified kernel fragments, it's all directed by the user.  Also, if you want to support non-contiguous arrays, you need to send the actual ``GPUArray`` objects, rather than their ``gpudata`` members to the call or prepare_ functions.\r\n\r\nAll existing tests succeed.  More would probably need to be added if it made sense to integrate this into PyCUDA.\r\n\r\nPositive side-effects: ``GPUArray.get()`` and ``GPUArray.copy()`` now work for arbitrarily sliced/strided arrays.\r\n\r\nThe performance hit for contiguous arrays is around 15% for modest-sized arrays (i.e. the 1000x100 array tested by Keegan in the above discussion) and, looking at profiling output, I think the hit is due to generating the key (in ``ElementwiseSourceModule.create_key()``) for hashing cached kernels.  This could probably be fixed.  Performance for non-contiguous arrays is _infinitely_ better, given that they weren't supported before, but I've seen a 40% slowdown over the contiguous version for the ``b1[::2,:-1]**2`` test Keegan tried, due to the need to calculate indexes at each iteration of the loop.  It tries to do this in a smart way, by pre-calculating the per-thread step for each dimension, and only using division/modulo to calculate the starting indices for each thread before the loop.\r\n\r\nIndependently of whether these changes are merged in, I will continue to use and develop them to support some local needs, so comments are welcome.  I hope this is useful!\r\n"},"comment":{"url":"https://api.github.com/repos/inducer/pycuda/issues/comments/367430312","html_url":"https://github.com/inducer/pycuda/pull/171#issuecomment-367430312","issue_url":"https://api.github.com/repos/inducer/pycuda/issues/171","id":367430312,"user":{"login":"SyamGadde","id":1247460,"avatar_url":"https://avatars1.githubusercontent.com/u/1247460?v=4","gravatar_id":"","url":"https://api.github.com/users/SyamGadde","html_url":"https://github.com/SyamGadde","followers_url":"https://api.github.com/users/SyamGadde/followers","following_url":"https://api.github.com/users/SyamGadde/following{/other_user}","gists_url":"https://api.github.com/users/SyamGadde/gists{/gist_id}","starred_url":"https://api.github.com/users/SyamGadde/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SyamGadde/subscriptions","organizations_url":"https://api.github.com/users/SyamGadde/orgs","repos_url":"https://api.github.com/users/SyamGadde/repos","events_url":"https://api.github.com/users/SyamGadde/events{/privacy}","received_events_url":"https://api.github.com/users/SyamGadde/received_events","type":"User","site_admin":false},"created_at":"2018-02-21T18:50:07Z","updated_at":"2018-02-21T18:50:07Z","author_association":"CONTRIBUTOR","body":"Also, wanted to reference the issues this patch may address:\r\n\r\n#15 \r\n#66 \r\n#121 \r\n#145 \r\n#151 \r\n#154 \r\n"}}
{"actor":{"display_login":"SyamGadde"}}