{"action":"created","comment":{"url":"https://api.github.com/repos/SciTools/iris/pulls/comments/173128470","pull_request_review_id":102262776,"id":173128470,"diff_hunk":"@@ -158,3 +178,18 @@ def multidim_lazy_stack(stack):\n         result = da.stack([multidim_lazy_stack(subarray)\n                            for subarray in stack])\n     return result\n+\n+\n+def co_realise_cubes(cubes):\n+    \"\"\"\n+    Fetch real data for multiple cubes at one time.\n+\n+    This fetches lazy content, equivalent to accessing each cube.data.\n+    However, lazy calculations and data fetches can be shared between the\n+    computations, improving performance.\n+\n+    \"\"\"\n+    results = _co_realise_lazy_arrays([cube.core_data() for cube in cubes])","path":"lib/iris/_lazy_data.py","position":71,"original_position":71,"commit_id":"4f2b2266a074bb5cebcd8282cb2e20e0ec3c250e","original_commit_id":"4f2b2266a074bb5cebcd8282cb2e20e0ec3c250e","user":{"login":"pp-mo","id":2089069,"avatar_url":"https://avatars1.githubusercontent.com/u/2089069?v=4","gravatar_id":"","url":"https://api.github.com/users/pp-mo","html_url":"https://github.com/pp-mo","followers_url":"https://api.github.com/users/pp-mo/followers","following_url":"https://api.github.com/users/pp-mo/following{/other_user}","gists_url":"https://api.github.com/users/pp-mo/gists{/gist_id}","starred_url":"https://api.github.com/users/pp-mo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pp-mo/subscriptions","organizations_url":"https://api.github.com/users/pp-mo/orgs","repos_url":"https://api.github.com/users/pp-mo/repos","events_url":"https://api.github.com/users/pp-mo/events{/privacy}","received_events_url":"https://api.github.com/users/pp-mo/received_events","type":"User","site_admin":false},"body":"You're right it absolutely was in my mind that it does do this.\r\nI will add a comment.","created_at":"2018-03-08T11:08:16Z","updated_at":"2018-03-08T11:08:16Z","html_url":"https://github.com/SciTools/iris/pull/2967#discussion_r173128470","pull_request_url":"https://api.github.com/repos/SciTools/iris/pulls/2967","author_association":"MEMBER","_links":{"self":{"href":"https://api.github.com/repos/SciTools/iris/pulls/comments/173128470"},"html":{"href":"https://github.com/SciTools/iris/pull/2967#discussion_r173128470"},"pull_request":{"href":"https://api.github.com/repos/SciTools/iris/pulls/2967"}},"in_reply_to_id":173125043},"pull_request":{"url":"https://api.github.com/repos/SciTools/iris/pulls/2967","id":173423383,"html_url":"https://github.com/SciTools/iris/pull/2967","diff_url":"https://github.com/SciTools/iris/pull/2967.diff","patch_url":"https://github.com/SciTools/iris/pull/2967.patch","issue_url":"https://api.github.com/repos/SciTools/iris/issues/2967","number":2967,"state":"open","locked":false,"title":"Add cube co-realisation.","user":{"login":"pp-mo","id":2089069,"avatar_url":"https://avatars1.githubusercontent.com/u/2089069?v=4","gravatar_id":"","url":"https://api.github.com/users/pp-mo","html_url":"https://github.com/pp-mo","followers_url":"https://api.github.com/users/pp-mo/followers","following_url":"https://api.github.com/users/pp-mo/following{/other_user}","gists_url":"https://api.github.com/users/pp-mo/gists{/gist_id}","starred_url":"https://api.github.com/users/pp-mo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pp-mo/subscriptions","organizations_url":"https://api.github.com/users/pp-mo/orgs","repos_url":"https://api.github.com/users/pp-mo/repos","events_url":"https://api.github.com/users/pp-mo/events{/privacy}","received_events_url":"https://api.github.com/users/pp-mo/received_events","type":"User","site_admin":false},"body":"User code often forms multiple stats, where data may be very large.\r\n= several small, lazy, calculated results derived from same large, lazy data \r\nCalculation scans the data, each time you realise one.\r\nThis can + should be done in parallel with `da.compute(*dask_stats_arrays)`.\r\nYou then need to assign those results back into Iris `cube.data` elements.\r\n\r\nI can't find a neat Python idiom for this, so we can provide a simple utility routine.\r\n\r\nNote: I'm proposing to put this in package `iris` itself, as I don't rate the alternatives :\r\n  * `iris._lazy_data` is private\r\n  * `iris.utils`, on inspection, I think is for \"handy things you might just have written yourself\"\r\n     *whereas* : this code uses knowledge of internals.\r\n","created_at":"2018-03-07T11:24:10Z","updated_at":"2018-03-08T11:08:16Z","closed_at":null,"merged_at":null,"merge_commit_sha":"3499a60e4575aa8422b22c9082ff3e39af83d85e","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/SciTools/iris/pulls/2967/commits","review_comments_url":"https://api.github.com/repos/SciTools/iris/pulls/2967/comments","review_comment_url":"https://api.github.com/repos/SciTools/iris/pulls/comments{/number}","comments_url":"https://api.github.com/repos/SciTools/iris/issues/2967/comments","statuses_url":"https://api.github.com/repos/SciTools/iris/statuses/4f2b2266a074bb5cebcd8282cb2e20e0ec3c250e","head":{"label":"pp-mo:co_realise","ref":"co_realise","sha":"4f2b2266a074bb5cebcd8282cb2e20e0ec3c250e","user":{"login":"pp-mo","id":2089069,"avatar_url":"https://avatars1.githubusercontent.com/u/2089069?v=4","gravatar_id":"","url":"https://api.github.com/users/pp-mo","html_url":"https://github.com/pp-mo","followers_url":"https://api.github.com/users/pp-mo/followers","following_url":"https://api.github.com/users/pp-mo/following{/other_user}","gists_url":"https://api.github.com/users/pp-mo/gists{/gist_id}","starred_url":"https://api.github.com/users/pp-mo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pp-mo/subscriptions","organizations_url":"https://api.github.com/users/pp-mo/orgs","repos_url":"https://api.github.com/users/pp-mo/repos","events_url":"https://api.github.com/users/pp-mo/events{/privacy}","received_events_url":"https://api.github.com/users/pp-mo/received_events","type":"User","site_admin":false},"repo":{"id":5701421,"name":"iris","full_name":"pp-mo/iris","owner":{"login":"pp-mo","id":2089069,"avatar_url":"https://avatars1.githubusercontent.com/u/2089069?v=4","gravatar_id":"","url":"https://api.github.com/users/pp-mo","html_url":"https://github.com/pp-mo","followers_url":"https://api.github.com/users/pp-mo/followers","following_url":"https://api.github.com/users/pp-mo/following{/other_user}","gists_url":"https://api.github.com/users/pp-mo/gists{/gist_id}","starred_url":"https://api.github.com/users/pp-mo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pp-mo/subscriptions","organizations_url":"https://api.github.com/users/pp-mo/orgs","repos_url":"https://api.github.com/users/pp-mo/repos","events_url":"https://api.github.com/users/pp-mo/events{/privacy}","received_events_url":"https://api.github.com/users/pp-mo/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/pp-mo/iris","description":null,"fork":true,"url":"https://api.github.com/repos/pp-mo/iris","forks_url":"https://api.github.com/repos/pp-mo/iris/forks","keys_url":"https://api.github.com/repos/pp-mo/iris/keys{/key_id}","collaborators_url":"https://api.github.com/repos/pp-mo/iris/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/pp-mo/iris/teams","hooks_url":"https://api.github.com/repos/pp-mo/iris/hooks","issue_events_url":"https://api.github.com/repos/pp-mo/iris/issues/events{/number}","events_url":"https://api.github.com/repos/pp-mo/iris/events","assignees_url":"https://api.github.com/repos/pp-mo/iris/assignees{/user}","branches_url":"https://api.github.com/repos/pp-mo/iris/branches{/branch}","tags_url":"https://api.github.com/repos/pp-mo/iris/tags","blobs_url":"https://api.github.com/repos/pp-mo/iris/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/pp-mo/iris/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/pp-mo/iris/git/refs{/sha}","trees_url":"https://api.github.com/repos/pp-mo/iris/git/trees{/sha}","statuses_url":"https://api.github.com/repos/pp-mo/iris/statuses/{sha}","languages_url":"https://api.github.com/repos/pp-mo/iris/languages","stargazers_url":"https://api.github.com/repos/pp-mo/iris/stargazers","contributors_url":"https://api.github.com/repos/pp-mo/iris/contributors","subscribers_url":"https://api.github.com/repos/pp-mo/iris/subscribers","subscription_url":"https://api.github.com/repos/pp-mo/iris/subscription","commits_url":"https://api.github.com/repos/pp-mo/iris/commits{/sha}","git_commits_url":"https://api.github.com/repos/pp-mo/iris/git/commits{/sha}","comments_url":"https://api.github.com/repos/pp-mo/iris/comments{/number}","issue_comment_url":"https://api.github.com/repos/pp-mo/iris/issues/comments{/number}","contents_url":"https://api.github.com/repos/pp-mo/iris/contents/{+path}","compare_url":"https://api.github.com/repos/pp-mo/iris/compare/{base}...{head}","merges_url":"https://api.github.com/repos/pp-mo/iris/merges","archive_url":"https://api.github.com/repos/pp-mo/iris/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/pp-mo/iris/downloads","issues_url":"https://api.github.com/repos/pp-mo/iris/issues{/number}","pulls_url":"https://api.github.com/repos/pp-mo/iris/pulls{/number}","milestones_url":"https://api.github.com/repos/pp-mo/iris/milestones{/number}","notifications_url":"https://api.github.com/repos/pp-mo/iris/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/pp-mo/iris/labels{/name}","releases_url":"https://api.github.com/repos/pp-mo/iris/releases{/id}","deployments_url":"https://api.github.com/repos/pp-mo/iris/deployments","created_at":"2012-09-06T11:45:58Z","updated_at":"2015-04-25T22:15:43Z","pushed_at":"2018-03-08T00:45:00Z","git_url":"git://github.com/pp-mo/iris.git","ssh_url":"git@github.com:pp-mo/iris.git","clone_url":"https://github.com/pp-mo/iris.git","svn_url":"https://github.com/pp-mo/iris","homepage":"http://scitools.github.com/iris","size":106368,"stargazers_count":0,"watchers_count":0,"language":"Python","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":5,"license":{"key":"lgpl-3.0","name":"GNU Lesser General Public License v3.0","spdx_id":"LGPL-3.0","url":"https://api.github.com/licenses/lgpl-3.0"},"forks":0,"open_issues":5,"watchers":0,"default_branch":"master"}},"base":{"label":"SciTools:master","ref":"master","sha":"089b14f48ab35d3ad41cc498f1156fc0fb6f580e","user":{"login":"SciTools","id":1391487,"avatar_url":"https://avatars3.githubusercontent.com/u/1391487?v=4","gravatar_id":"","url":"https://api.github.com/users/SciTools","html_url":"https://github.com/SciTools","followers_url":"https://api.github.com/users/SciTools/followers","following_url":"https://api.github.com/users/SciTools/following{/other_user}","gists_url":"https://api.github.com/users/SciTools/gists{/gist_id}","starred_url":"https://api.github.com/users/SciTools/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SciTools/subscriptions","organizations_url":"https://api.github.com/users/SciTools/orgs","repos_url":"https://api.github.com/users/SciTools/repos","events_url":"https://api.github.com/users/SciTools/events{/privacy}","received_events_url":"https://api.github.com/users/SciTools/received_events","type":"Organization","site_admin":false},"repo":{"id":5312648,"name":"iris","full_name":"SciTools/iris","owner":{"login":"SciTools","id":1391487,"avatar_url":"https://avatars3.githubusercontent.com/u/1391487?v=4","gravatar_id":"","url":"https://api.github.com/users/SciTools","html_url":"https://github.com/SciTools","followers_url":"https://api.github.com/users/SciTools/followers","following_url":"https://api.github.com/users/SciTools/following{/other_user}","gists_url":"https://api.github.com/users/SciTools/gists{/gist_id}","starred_url":"https://api.github.com/users/SciTools/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SciTools/subscriptions","organizations_url":"https://api.github.com/users/SciTools/orgs","repos_url":"https://api.github.com/users/SciTools/repos","events_url":"https://api.github.com/users/SciTools/events{/privacy}","received_events_url":"https://api.github.com/users/SciTools/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/SciTools/iris","description":"A powerful, easy to use, and community-driven Python library for analysing and visualising meteorological and oceanographic data sets.","fork":false,"url":"https://api.github.com/repos/SciTools/iris","forks_url":"https://api.github.com/repos/SciTools/iris/forks","keys_url":"https://api.github.com/repos/SciTools/iris/keys{/key_id}","collaborators_url":"https://api.github.com/repos/SciTools/iris/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/SciTools/iris/teams","hooks_url":"https://api.github.com/repos/SciTools/iris/hooks","issue_events_url":"https://api.github.com/repos/SciTools/iris/issues/events{/number}","events_url":"https://api.github.com/repos/SciTools/iris/events","assignees_url":"https://api.github.com/repos/SciTools/iris/assignees{/user}","branches_url":"https://api.github.com/repos/SciTools/iris/branches{/branch}","tags_url":"https://api.github.com/repos/SciTools/iris/tags","blobs_url":"https://api.github.com/repos/SciTools/iris/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/SciTools/iris/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/SciTools/iris/git/refs{/sha}","trees_url":"https://api.github.com/repos/SciTools/iris/git/trees{/sha}","statuses_url":"https://api.github.com/repos/SciTools/iris/statuses/{sha}","languages_url":"https://api.github.com/repos/SciTools/iris/languages","stargazers_url":"https://api.github.com/repos/SciTools/iris/stargazers","contributors_url":"https://api.github.com/repos/SciTools/iris/contributors","subscribers_url":"https://api.github.com/repos/SciTools/iris/subscribers","subscription_url":"https://api.github.com/repos/SciTools/iris/subscription","commits_url":"https://api.github.com/repos/SciTools/iris/commits{/sha}","git_commits_url":"https://api.github.com/repos/SciTools/iris/git/commits{/sha}","comments_url":"https://api.github.com/repos/SciTools/iris/comments{/number}","issue_comment_url":"https://api.github.com/repos/SciTools/iris/issues/comments{/number}","contents_url":"https://api.github.com/repos/SciTools/iris/contents/{+path}","compare_url":"https://api.github.com/repos/SciTools/iris/compare/{base}...{head}","merges_url":"https://api.github.com/repos/SciTools/iris/merges","archive_url":"https://api.github.com/repos/SciTools/iris/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/SciTools/iris/downloads","issues_url":"https://api.github.com/repos/SciTools/iris/issues{/number}","pulls_url":"https://api.github.com/repos/SciTools/iris/pulls{/number}","milestones_url":"https://api.github.com/repos/SciTools/iris/milestones{/number}","notifications_url":"https://api.github.com/repos/SciTools/iris/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/SciTools/iris/labels{/name}","releases_url":"https://api.github.com/repos/SciTools/iris/releases{/id}","deployments_url":"https://api.github.com/repos/SciTools/iris/deployments","created_at":"2012-08-06T10:09:14Z","updated_at":"2018-03-01T03:43:03Z","pushed_at":"2018-03-08T00:45:01Z","git_url":"git://github.com/SciTools/iris.git","ssh_url":"git@github.com:SciTools/iris.git","clone_url":"https://github.com/SciTools/iris.git","svn_url":"https://github.com/SciTools/iris","homepage":"http://scitools.org.uk/iris/","size":104598,"stargazers_count":207,"watchers_count":207,"language":"Python","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":139,"mirror_url":null,"archived":false,"open_issues_count":220,"license":{"key":"lgpl-3.0","name":"GNU Lesser General Public License v3.0","spdx_id":"LGPL-3.0","url":"https://api.github.com/licenses/lgpl-3.0"},"forks":139,"open_issues":220,"watchers":207,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/SciTools/iris/pulls/2967"},"html":{"href":"https://github.com/SciTools/iris/pull/2967"},"issue":{"href":"https://api.github.com/repos/SciTools/iris/issues/2967"},"comments":{"href":"https://api.github.com/repos/SciTools/iris/issues/2967/comments"},"review_comments":{"href":"https://api.github.com/repos/SciTools/iris/pulls/2967/comments"},"review_comment":{"href":"https://api.github.com/repos/SciTools/iris/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/SciTools/iris/pulls/2967/commits"},"statuses":{"href":"https://api.github.com/repos/SciTools/iris/statuses/4f2b2266a074bb5cebcd8282cb2e20e0ec3c250e"}},"author_association":"MEMBER"}}
{"actor":{"display_login":"pp-mo"}}