{"action":"created","issue":{"url":"https://api.github.com/repos/dask/dask/issues/3586","repository_url":"https://api.github.com/repos/dask/dask","labels_url":"https://api.github.com/repos/dask/dask/issues/3586/labels{/name}","comments_url":"https://api.github.com/repos/dask/dask/issues/3586/comments","events_url":"https://api.github.com/repos/dask/dask/issues/3586/events","html_url":"https://github.com/dask/dask/pull/3586","id":330972651,"node_id":"MDExOlB1bGxSZXF1ZXN0MTkzODI4NzEw","number":3586,"title":"Added recursion to array/linalg/tsqr to better manage the single core bottleneck","user":{"login":"convexset","id":1677933,"node_id":"MDQ6VXNlcjE2Nzc5MzM=","avatar_url":"https://avatars1.githubusercontent.com/u/1677933?v=4","gravatar_id":"","url":"https://api.github.com/users/convexset","html_url":"https://github.com/convexset","followers_url":"https://api.github.com/users/convexset/followers","following_url":"https://api.github.com/users/convexset/following{/other_user}","gists_url":"https://api.github.com/users/convexset/gists{/gist_id}","starred_url":"https://api.github.com/users/convexset/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/convexset/subscriptions","organizations_url":"https://api.github.com/users/convexset/orgs","repos_url":"https://api.github.com/users/convexset/repos","events_url":"https://api.github.com/users/convexset/events{/privacy}","received_events_url":"https://api.github.com/users/convexset/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-06-10T14:22:08Z","updated_at":"2018-06-11T12:05:13Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/dask/dask/pulls/3586","html_url":"https://github.com/dask/dask/pull/3586","diff_url":"https://github.com/dask/dask/pull/3586.diff","patch_url":"https://github.com/dask/dask/pull/3586.patch"},"body":"Added recursion to tsqr to better manage the single core bottleneck.\r\n\r\nFor example: a (200000 × (10000, 100)) tower of blocks will end up with a (200000×100, 100) QR decomposition to compute in the single core case. This would be significantly bigger than the expectation of having to do computes on (10000, 100) blocks. We can lighten that load by turning that single core step into a (20000 × (1000, 100)) recursive call, which becomes (2000 × (1000, 100)) then  (200 × (1000, 100)) then  (20 × (1000, 100)) then (2 × (1000, 100)) then a single core  (200, 100) QR compute.\r\n\r\nMore elaboration in: https://github.com/dask/dask/issues/3584\r\n\r\nA balance is struck by only recursing when there is a need for it (the single core compute has to work on a block/chunk larger than the largest chunk) and when \"meaningful reduction can be made\" in the size of the single core compute (aspect ratio of the tall-and-skinny chunks is at least 2; the above example has an aspect ratio of 10).\r\n\r\n- [X] A few tests added (various recursion depths)\r\n- [X] Existing tests passed\r\n- [X] Passes `flake8 dask`"},"comment":{"url":"https://api.github.com/repos/dask/dask/issues/comments/396220126","html_url":"https://github.com/dask/dask/pull/3586#issuecomment-396220126","issue_url":"https://api.github.com/repos/dask/dask/issues/3586","id":396220126,"node_id":"MDEyOklzc3VlQ29tbWVudDM5NjIyMDEyNg==","user":{"login":"mrocklin","id":306380,"node_id":"MDQ6VXNlcjMwNjM4MA==","avatar_url":"https://avatars3.githubusercontent.com/u/306380?v=4","gravatar_id":"","url":"https://api.github.com/users/mrocklin","html_url":"https://github.com/mrocklin","followers_url":"https://api.github.com/users/mrocklin/followers","following_url":"https://api.github.com/users/mrocklin/following{/other_user}","gists_url":"https://api.github.com/users/mrocklin/gists{/gist_id}","starred_url":"https://api.github.com/users/mrocklin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mrocklin/subscriptions","organizations_url":"https://api.github.com/users/mrocklin/orgs","repos_url":"https://api.github.com/users/mrocklin/repos","events_url":"https://api.github.com/users/mrocklin/events{/privacy}","received_events_url":"https://api.github.com/users/mrocklin/received_events","type":"User","site_admin":false},"created_at":"2018-06-11T12:05:13Z","updated_at":"2018-06-11T12:05:13Z","author_association":"MEMBER","body":"This appears to conflict with master.  Depending on how familiar you are with git you might want to rebase and remove the old commits from the previous PR."}}
{"actor":{"display_login":"mrocklin"}}