{"action":"created","issue":{"url":"https://api.github.com/repos/dask/distributed/issues/183","repository_url":"https://api.github.com/repos/dask/distributed","labels_url":"https://api.github.com/repos/dask/distributed/issues/183/labels{/name}","comments_url":"https://api.github.com/repos/dask/distributed/issues/183/comments","events_url":"https://api.github.com/repos/dask/distributed/issues/183/events","html_url":"https://github.com/dask/distributed/pull/183","id":140743315,"number":183,"title":"Rebalance","user":{"login":"mrocklin","id":306380,"avatar_url":"https://avatars.githubusercontent.com/u/306380?v=3","gravatar_id":"","url":"https://api.github.com/users/mrocklin","html_url":"https://github.com/mrocklin","followers_url":"https://api.github.com/users/mrocklin/followers","following_url":"https://api.github.com/users/mrocklin/following{/other_user}","gists_url":"https://api.github.com/users/mrocklin/gists{/gist_id}","starred_url":"https://api.github.com/users/mrocklin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mrocklin/subscriptions","organizations_url":"https://api.github.com/users/mrocklin/orgs","repos_url":"https://api.github.com/users/mrocklin/repos","events_url":"https://api.github.com/users/mrocklin/events{/privacy}","received_events_url":"https://api.github.com/users/mrocklin/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2016-03-14T17:35:42Z","updated_at":"2016-03-15T21:18:10Z","closed_at":"2016-03-14T22:13:06Z","pull_request":{"url":"https://api.github.com/repos/dask/distributed/pulls/183","html_url":"https://github.com/dask/distributed/pull/183","diff_url":"https://github.com/dask/distributed/pull/183.diff","patch_url":"https://github.com/dask/distributed/pull/183.patch"},"body":"This adds a rebalance operation to the Scheduler and Executor to balance the data on the cluster so that workers share roughly equal numbers of bytes (as measured by `sizeof`).\r\n\r\nYou can optionally specify a set of futures and a set of workers to balance.\r\n\r\n```python\r\nIn [1]: from distributed import Executor\r\n\r\nIn [2]: e = Executor('192.168.1.141:8786')\r\n\r\nIn [3]: e\r\nOut[3]: <Executor: scheduler=192.168.1.141:8786 workers=2 threads=16>\r\n\r\nIn [4]: e.ncores()\r\nOut[4]: {'192.168.1.141:49293': 8, '192.168.1.141:57518': 8}\r\n\r\nIn [5]: def inc(x):\r\n    return x + 1\r\n   ...: \r\n\r\nIn [6]: futures = e.map(inc, range(10), workers='192.168.1.141:49293')\r\n\r\nIn [7]: e.has_what()\r\nOut[7]: \r\n{'192.168.1.141:49293': ['inc-2331da85e6990c120becd8b2cb613a1a',\r\n  'inc-249b8519655b9c9d20ca38791d8b5884',\r\n  'inc-6891d0d77b3127c8a26b3baa27e7ab67',\r\n  'inc-52cc7ada53ca3e3474243fd7e647c96a',\r\n  'inc-ec040381c2aa31951cb497269b0f2eb9',\r\n  'inc-1db43ecb22d0d8830f529e6ba7beef73',\r\n  'inc-e6796c5cab6344970f156ffe644404aa',\r\n  'inc-42b0e6067d724019cbabb3aa7a7d594d',\r\n  'inc-7865dd82fe814767dd48d67d5b0d67b3',\r\n  'inc-642e5d7d079ddbf1840d4d9a6849f97f'],\r\n '192.168.1.141:57518': []}\r\n\r\nIn [8]: e.rebalance()\r\nOut[8]: (None,)\r\n\r\nIn [9]: e.has_what()\r\nOut[9]: \r\n{'192.168.1.141:49293': ['inc-6891d0d77b3127c8a26b3baa27e7ab67',\r\n  'inc-ec040381c2aa31951cb497269b0f2eb9',\r\n  'inc-1db43ecb22d0d8830f529e6ba7beef73',\r\n  'inc-7865dd82fe814767dd48d67d5b0d67b3',\r\n  'inc-642e5d7d079ddbf1840d4d9a6849f97f'],\r\n '192.168.1.141:57518': ['inc-2331da85e6990c120becd8b2cb613a1a',\r\n  'inc-e6796c5cab6344970f156ffe644404aa',\r\n  'inc-249b8519655b9c9d20ca38791d8b5884',\r\n  'inc-42b0e6067d724019cbabb3aa7a7d594d',\r\n  'inc-52cc7ada53ca3e3474243fd7e647c96a']}\r\n```\r\n\r\ncc @sklam"},"comment":{"url":"https://api.github.com/repos/dask/distributed/issues/comments/197024103","html_url":"https://github.com/dask/distributed/pull/183#issuecomment-197024103","issue_url":"https://api.github.com/repos/dask/distributed/issues/183","id":197024103,"user":{"login":"sklam","id":1929845,"avatar_url":"https://avatars.githubusercontent.com/u/1929845?v=3","gravatar_id":"","url":"https://api.github.com/users/sklam","html_url":"https://github.com/sklam","followers_url":"https://api.github.com/users/sklam/followers","following_url":"https://api.github.com/users/sklam/following{/other_user}","gists_url":"https://api.github.com/users/sklam/gists{/gist_id}","starred_url":"https://api.github.com/users/sklam/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sklam/subscriptions","organizations_url":"https://api.github.com/users/sklam/orgs","repos_url":"https://api.github.com/users/sklam/repos","events_url":"https://api.github.com/users/sklam/events{/privacy}","received_events_url":"https://api.github.com/users/sklam/received_events","type":"User","site_admin":false},"created_at":"2016-03-15T21:18:10Z","updated_at":"2016-03-15T21:18:10Z","body":"This is perfect.  For my application, I found it most useful with ``e.rebalance(futures=images)``."}}