{"action":"created","issue":{"url":"https://api.github.com/repos/pydata/xarray/issues/1837","repository_url":"https://api.github.com/repos/pydata/xarray","labels_url":"https://api.github.com/repos/pydata/xarray/issues/1837/labels{/name}","comments_url":"https://api.github.com/repos/pydata/xarray/issues/1837/comments","events_url":"https://api.github.com/repos/pydata/xarray/issues/1837/events","html_url":"https://github.com/pydata/xarray/pull/1837","id":289556132,"number":1837,"title":"Rolling window with `as_strided`","user":{"login":"fujiisoup","id":6815844,"avatar_url":"https://avatars2.githubusercontent.com/u/6815844?v=4","gravatar_id":"","url":"https://api.github.com/users/fujiisoup","html_url":"https://github.com/fujiisoup","followers_url":"https://api.github.com/users/fujiisoup/followers","following_url":"https://api.github.com/users/fujiisoup/following{/other_user}","gists_url":"https://api.github.com/users/fujiisoup/gists{/gist_id}","starred_url":"https://api.github.com/users/fujiisoup/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fujiisoup/subscriptions","organizations_url":"https://api.github.com/users/fujiisoup/orgs","repos_url":"https://api.github.com/users/fujiisoup/repos","events_url":"https://api.github.com/users/fujiisoup/events{/privacy}","received_events_url":"https://api.github.com/users/fujiisoup/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2018-01-18T09:18:19Z","updated_at":"2018-02-07T01:11:12Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/pydata/xarray/pulls/1837","html_url":"https://github.com/pydata/xarray/pull/1837","diff_url":"https://github.com/pydata/xarray/pull/1837.diff","patch_url":"https://github.com/pydata/xarray/pull/1837.patch"},"body":" - [x] Closes #1831, #1142, #819\r\n - [x] Tests added\r\n - [x] Tests passed\r\n - [x] Passes ``git diff upstream/master **/*py | flake8 --diff``\r\n - [x] Fully documented, including `whats-new.rst` for all changes and `api.rst` for new API\r\n\r\nI started to work for refactoring rollings.\r\nAs suggested in [#1831 comment](https://github.com/pydata/xarray/issues/1831#issuecomment-357828636), I implemented `rolling_window` methods based on `as_strided`.\r\n\r\nI got more than 1,000 times speed up! yey!\r\n\r\n```python\r\nIn [1]: import numpy as np\r\n   ...: import xarray as xr\r\n   ...: \r\n   ...: da = xr.DataArray(np.random.randn(10000, 3), dims=['x', 'y'])\r\n```\r\nwith the master\r\n```python\r\n%timeit da.rolling(x=5).reduce(np.mean)\r\n1 loop, best of 3: 9.68 s per loop\r\n```\r\nwith the current implementation\r\n```python\r\n%timeit da.rolling(x=5).reduce(np.mean)\r\n100 loops, best of 3: 5.29 ms per loop\r\n```\r\nand with the bottleneck\r\n```python\r\n%timeit da.rolling(x=5).mean()\r\n100 loops, best of 3: 2.62 ms per loop\r\n```\r\n\r\nMy current concerns are \r\n+ Can we expose the new `rolling_window` method of `DataArray` and `Dataset` to the public?\r\nI think this method itself is useful for many usecases, such as short-term-FFT and convolution.\r\nThis also gives more flexible rolling operation, such as windowed moving average, strided rolling, and ND-rolling.\r\n\r\n+ Is there any dask's equivalence to numpy's `as_strided`?\r\nCurrently, I just use a slice->concatenate path, but I don't think it is very efficient.\r\n(Is it already efficient, as dask utilizes out-of-core computation?)\r\n\r\nAny thoughts are welcome.\r\n"},"comment":{"url":"https://api.github.com/repos/pydata/xarray/issues/comments/363621372","html_url":"https://github.com/pydata/xarray/pull/1837#issuecomment-363621372","issue_url":"https://api.github.com/repos/pydata/xarray/issues/1837","id":363621372,"user":{"login":"fujiisoup","id":6815844,"avatar_url":"https://avatars2.githubusercontent.com/u/6815844?v=4","gravatar_id":"","url":"https://api.github.com/users/fujiisoup","html_url":"https://github.com/fujiisoup","followers_url":"https://api.github.com/users/fujiisoup/followers","following_url":"https://api.github.com/users/fujiisoup/following{/other_user}","gists_url":"https://api.github.com/users/fujiisoup/gists{/gist_id}","starred_url":"https://api.github.com/users/fujiisoup/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fujiisoup/subscriptions","organizations_url":"https://api.github.com/users/fujiisoup/orgs","repos_url":"https://api.github.com/users/fujiisoup/repos","events_url":"https://api.github.com/users/fujiisoup/events{/privacy}","received_events_url":"https://api.github.com/users/fujiisoup/received_events","type":"User","site_admin":false},"created_at":"2018-02-07T01:11:12Z","updated_at":"2018-02-07T01:11:12Z","author_association":"CONTRIBUTOR","body":"I sent a separate PR (#1883) for nan-reduce methods.\r\nI will continue / finish up this PR after #1883 is merged."}}
{"actor":{"display_login":"fujiisoup"}}