{"action":"created","comment":{"url":"https://api.github.com/repos/amplab/ray/pulls/comments/71794331","id":71794331,"diff_hunk":"@@ -553,6 +661,20 @@ def _mode(worker=global_worker):\n   \"\"\"\n   return worker.mode\n \n+def _export_reusable_variable(name, initializer, reinitializer, worker=global_worker):\n+  \"\"\"Export a reusable variable to the workers. This is only called by a driver.\n+\n+  Args:\n+    name (str): The name of the variable to export.\n+    initializer (Callable[[], object]): The function for initializing the\n+      reusable variable.\n+    reinitializer (Callable[[object], object]): The function for reinitializing\n+      the reusable variable.\n+  \"\"\"\n+  if _mode() not in [ray.SHELL_MODE, ray.SCRIPT_MODE]:\n+    raise Exception(\"_export_reusable_variable can only be called on a driver.\")\n+  ray.lib.export_reusable_variable(worker.handle, name, pickling.serialize(initializer), pickling.serialize(reinitializer))","path":"lib/python/ray/worker.py","position":171,"original_position":171,"commit_id":"901639dfa2df4238f089254203e21d32dec8c636","original_commit_id":"901639dfa2df4238f089254203e21d32dec8c636","user":{"login":"mehrdadn","id":523212,"avatar_url":"https://avatars.githubusercontent.com/u/523212?v=3","gravatar_id":"","url":"https://api.github.com/users/mehrdadn","html_url":"https://github.com/mehrdadn","followers_url":"https://api.github.com/users/mehrdadn/followers","following_url":"https://api.github.com/users/mehrdadn/following{/other_user}","gists_url":"https://api.github.com/users/mehrdadn/gists{/gist_id}","starred_url":"https://api.github.com/users/mehrdadn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mehrdadn/subscriptions","organizations_url":"https://api.github.com/users/mehrdadn/orgs","repos_url":"https://api.github.com/users/mehrdadn/repos","events_url":"https://api.github.com/users/mehrdadn/events{/privacy}","received_events_url":"https://api.github.com/users/mehrdadn/received_events","type":"User","site_admin":false},"body":"I would check to see if the two functions are the same (at least an `is` test), and if so, only pickle them once. Pickling is very expensive.","created_at":"2016-07-21T21:52:02Z","updated_at":"2016-07-21T21:52:02Z","html_url":"https://github.com/amplab/ray/pull/283#discussion_r71794331","pull_request_url":"https://api.github.com/repos/amplab/ray/pulls/283","_links":{"self":{"href":"https://api.github.com/repos/amplab/ray/pulls/comments/71794331"},"html":{"href":"https://github.com/amplab/ray/pull/283#discussion_r71794331"},"pull_request":{"href":"https://api.github.com/repos/amplab/ray/pulls/283"}}},"pull_request":{"url":"https://api.github.com/repos/amplab/ray/pulls/283","id":78391250,"html_url":"https://github.com/amplab/ray/pull/283","diff_url":"https://github.com/amplab/ray/pull/283.diff","patch_url":"https://github.com/amplab/ray/pull/283.patch","issue_url":"https://api.github.com/repos/amplab/ray/issues/283","number":283,"state":"open","locked":false,"title":"[WIP] implement key value store for sharing reusable variables","user":{"login":"robertnishihara","id":249517,"avatar_url":"https://avatars.githubusercontent.com/u/249517?v=3","gravatar_id":"","url":"https://api.github.com/users/robertnishihara","html_url":"https://github.com/robertnishihara","followers_url":"https://api.github.com/users/robertnishihara/followers","following_url":"https://api.github.com/users/robertnishihara/following{/other_user}","gists_url":"https://api.github.com/users/robertnishihara/gists{/gist_id}","starred_url":"https://api.github.com/users/robertnishihara/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/robertnishihara/subscriptions","organizations_url":"https://api.github.com/users/robertnishihara/orgs","repos_url":"https://api.github.com/users/robertnishihara/repos","events_url":"https://api.github.com/users/robertnishihara/events{/privacy}","received_events_url":"https://api.github.com/users/robertnishihara/received_events","type":"User","site_admin":false},"body":"Copied from the tests. You can use this code as follows (in `python scripts/shell.py`):\r\n\r\n```python\r\ndef foo_initializer():\r\n  return 1\r\ndef foo_reinitializer(foo):\r\n  return foo\r\n\r\nray.reusables.foo = foo_initializer, foo_reinitializer\r\n\r\n@ray.remote([], [int])\r\ndef use_foo():\r\n  return ray.reusables.foo\r\n\r\nassert ray.reusables.foo == 1\r\nassert ray.get(use_foo()) == 1\r\nassert ray.get(use_foo()) == 1\r\nassert ray.get(use_foo()) == 1\r\n\r\n# Test that we can add a variable to the key-value store, mutate it, and reset it.\r\n\r\ndef bar_initializer():\r\n  return [1, 2, 3]\r\n\r\nray.reusables.bar = bar_initializer\r\n\r\n@ray.remote([], [list])\r\ndef use_bar():\r\n  ray.reusables.bar.append(4)\r\n  return ray.reusables.bar\r\n\r\nassert ray.get(use_bar()) == [1, 2, 3, 4]\r\nassert ray.get(use_bar()) == [1, 2, 3, 4]\r\nassert ray.get(use_bar()) == [1, 2, 3, 4]\r\n\r\n# Test that we can use the reinitializer.\r\n\r\ndef baz_initializer():\r\n  return np.zeros([4])\r\ndef baz_reinitializer(baz):\r\n  for i in range(len(baz)):\r\n    baz[i] = 0\r\n  return baz\r\n\r\nray.reusables.baz = baz_initializer, baz_reinitializer\r\n\r\n@ray.remote([int], [np.ndarray])\r\ndef use_baz(i):\r\n  baz = ray.reusables.baz\r\n  baz[i] = 1\r\n  return baz\r\n\r\nassert np.alltrue(ray.get(use_baz(0)) == np.array([1, 0, 0, 0]))\r\nassert np.alltrue(ray.get(use_baz(1)) == np.array([0, 1, 0, 0]))\r\nassert np.alltrue(ray.get(use_baz(2)) == np.array([0, 0, 1, 0]))\r\nassert np.alltrue(ray.get(use_baz(3)) == np.array([0, 0, 0, 1]))\r\n```","created_at":"2016-07-21T21:32:21Z","updated_at":"2016-07-21T21:52:02Z","closed_at":null,"merged_at":null,"merge_commit_sha":"97b8c85daec3ae67b9200bfeb0c2673be9e4b31a","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/amplab/ray/pulls/283/commits","review_comments_url":"https://api.github.com/repos/amplab/ray/pulls/283/comments","review_comment_url":"https://api.github.com/repos/amplab/ray/pulls/comments{/number}","comments_url":"https://api.github.com/repos/amplab/ray/issues/283/comments","statuses_url":"https://api.github.com/repos/amplab/ray/statuses/901639dfa2df4238f089254203e21d32dec8c636","head":{"label":"amplab:kv","ref":"kv","sha":"901639dfa2df4238f089254203e21d32dec8c636","user":{"login":"amplab","id":1438454,"avatar_url":"https://avatars.githubusercontent.com/u/1438454?v=3","gravatar_id":"","url":"https://api.github.com/users/amplab","html_url":"https://github.com/amplab","followers_url":"https://api.github.com/users/amplab/followers","following_url":"https://api.github.com/users/amplab/following{/other_user}","gists_url":"https://api.github.com/users/amplab/gists{/gist_id}","starred_url":"https://api.github.com/users/amplab/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/amplab/subscriptions","organizations_url":"https://api.github.com/users/amplab/orgs","repos_url":"https://api.github.com/users/amplab/repos","events_url":"https://api.github.com/users/amplab/events{/privacy}","received_events_url":"https://api.github.com/users/amplab/received_events","type":"Organization","site_admin":false},"repo":{"id":51269228,"name":"ray","full_name":"amplab/ray","owner":{"login":"amplab","id":1438454,"avatar_url":"https://avatars.githubusercontent.com/u/1438454?v=3","gravatar_id":"","url":"https://api.github.com/users/amplab","html_url":"https://github.com/amplab","followers_url":"https://api.github.com/users/amplab/followers","following_url":"https://api.github.com/users/amplab/following{/other_user}","gists_url":"https://api.github.com/users/amplab/gists{/gist_id}","starred_url":"https://api.github.com/users/amplab/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/amplab/subscriptions","organizations_url":"https://api.github.com/users/amplab/orgs","repos_url":"https://api.github.com/users/amplab/repos","events_url":"https://api.github.com/users/amplab/events{/privacy}","received_events_url":"https://api.github.com/users/amplab/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/amplab/ray","description":"This is an experimental distributed execution engine","fork":false,"url":"https://api.github.com/repos/amplab/ray","forks_url":"https://api.github.com/repos/amplab/ray/forks","keys_url":"https://api.github.com/repos/amplab/ray/keys{/key_id}","collaborators_url":"https://api.github.com/repos/amplab/ray/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/amplab/ray/teams","hooks_url":"https://api.github.com/repos/amplab/ray/hooks","issue_events_url":"https://api.github.com/repos/amplab/ray/issues/events{/number}","events_url":"https://api.github.com/repos/amplab/ray/events","assignees_url":"https://api.github.com/repos/amplab/ray/assignees{/user}","branches_url":"https://api.github.com/repos/amplab/ray/branches{/branch}","tags_url":"https://api.github.com/repos/amplab/ray/tags","blobs_url":"https://api.github.com/repos/amplab/ray/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/amplab/ray/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/amplab/ray/git/refs{/sha}","trees_url":"https://api.github.com/repos/amplab/ray/git/trees{/sha}","statuses_url":"https://api.github.com/repos/amplab/ray/statuses/{sha}","languages_url":"https://api.github.com/repos/amplab/ray/languages","stargazers_url":"https://api.github.com/repos/amplab/ray/stargazers","contributors_url":"https://api.github.com/repos/amplab/ray/contributors","subscribers_url":"https://api.github.com/repos/amplab/ray/subscribers","subscription_url":"https://api.github.com/repos/amplab/ray/subscription","commits_url":"https://api.github.com/repos/amplab/ray/commits{/sha}","git_commits_url":"https://api.github.com/repos/amplab/ray/git/commits{/sha}","comments_url":"https://api.github.com/repos/amplab/ray/comments{/number}","issue_comment_url":"https://api.github.com/repos/amplab/ray/issues/comments{/number}","contents_url":"https://api.github.com/repos/amplab/ray/contents/{+path}","compare_url":"https://api.github.com/repos/amplab/ray/compare/{base}...{head}","merges_url":"https://api.github.com/repos/amplab/ray/merges","archive_url":"https://api.github.com/repos/amplab/ray/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/amplab/ray/downloads","issues_url":"https://api.github.com/repos/amplab/ray/issues{/number}","pulls_url":"https://api.github.com/repos/amplab/ray/pulls{/number}","milestones_url":"https://api.github.com/repos/amplab/ray/milestones{/number}","notifications_url":"https://api.github.com/repos/amplab/ray/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/amplab/ray/labels{/name}","releases_url":"https://api.github.com/repos/amplab/ray/releases{/id}","deployments_url":"https://api.github.com/repos/amplab/ray/deployments","created_at":"2016-02-07T22:18:40Z","updated_at":"2016-07-20T01:38:37Z","pushed_at":"2016-07-21T21:32:22Z","git_url":"git://github.com/amplab/ray.git","ssh_url":"git@github.com:amplab/ray.git","clone_url":"https://github.com/amplab/ray.git","svn_url":"https://github.com/amplab/ray","homepage":"","size":915,"stargazers_count":6,"watchers_count":6,"language":"Python","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":2,"mirror_url":null,"open_issues_count":36,"forks":2,"open_issues":36,"watchers":6,"default_branch":"master"}},"base":{"label":"amplab:master","ref":"master","sha":"b68512ec7db523b4857d015c153ae7dfa8d7af70","user":{"login":"amplab","id":1438454,"avatar_url":"https://avatars.githubusercontent.com/u/1438454?v=3","gravatar_id":"","url":"https://api.github.com/users/amplab","html_url":"https://github.com/amplab","followers_url":"https://api.github.com/users/amplab/followers","following_url":"https://api.github.com/users/amplab/following{/other_user}","gists_url":"https://api.github.com/users/amplab/gists{/gist_id}","starred_url":"https://api.github.com/users/amplab/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/amplab/subscriptions","organizations_url":"https://api.github.com/users/amplab/orgs","repos_url":"https://api.github.com/users/amplab/repos","events_url":"https://api.github.com/users/amplab/events{/privacy}","received_events_url":"https://api.github.com/users/amplab/received_events","type":"Organization","site_admin":false},"repo":{"id":51269228,"name":"ray","full_name":"amplab/ray","owner":{"login":"amplab","id":1438454,"avatar_url":"https://avatars.githubusercontent.com/u/1438454?v=3","gravatar_id":"","url":"https://api.github.com/users/amplab","html_url":"https://github.com/amplab","followers_url":"https://api.github.com/users/amplab/followers","following_url":"https://api.github.com/users/amplab/following{/other_user}","gists_url":"https://api.github.com/users/amplab/gists{/gist_id}","starred_url":"https://api.github.com/users/amplab/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/amplab/subscriptions","organizations_url":"https://api.github.com/users/amplab/orgs","repos_url":"https://api.github.com/users/amplab/repos","events_url":"https://api.github.com/users/amplab/events{/privacy}","received_events_url":"https://api.github.com/users/amplab/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/amplab/ray","description":"This is an experimental distributed execution engine","fork":false,"url":"https://api.github.com/repos/amplab/ray","forks_url":"https://api.github.com/repos/amplab/ray/forks","keys_url":"https://api.github.com/repos/amplab/ray/keys{/key_id}","collaborators_url":"https://api.github.com/repos/amplab/ray/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/amplab/ray/teams","hooks_url":"https://api.github.com/repos/amplab/ray/hooks","issue_events_url":"https://api.github.com/repos/amplab/ray/issues/events{/number}","events_url":"https://api.github.com/repos/amplab/ray/events","assignees_url":"https://api.github.com/repos/amplab/ray/assignees{/user}","branches_url":"https://api.github.com/repos/amplab/ray/branches{/branch}","tags_url":"https://api.github.com/repos/amplab/ray/tags","blobs_url":"https://api.github.com/repos/amplab/ray/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/amplab/ray/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/amplab/ray/git/refs{/sha}","trees_url":"https://api.github.com/repos/amplab/ray/git/trees{/sha}","statuses_url":"https://api.github.com/repos/amplab/ray/statuses/{sha}","languages_url":"https://api.github.com/repos/amplab/ray/languages","stargazers_url":"https://api.github.com/repos/amplab/ray/stargazers","contributors_url":"https://api.github.com/repos/amplab/ray/contributors","subscribers_url":"https://api.github.com/repos/amplab/ray/subscribers","subscription_url":"https://api.github.com/repos/amplab/ray/subscription","commits_url":"https://api.github.com/repos/amplab/ray/commits{/sha}","git_commits_url":"https://api.github.com/repos/amplab/ray/git/commits{/sha}","comments_url":"https://api.github.com/repos/amplab/ray/comments{/number}","issue_comment_url":"https://api.github.com/repos/amplab/ray/issues/comments{/number}","contents_url":"https://api.github.com/repos/amplab/ray/contents/{+path}","compare_url":"https://api.github.com/repos/amplab/ray/compare/{base}...{head}","merges_url":"https://api.github.com/repos/amplab/ray/merges","archive_url":"https://api.github.com/repos/amplab/ray/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/amplab/ray/downloads","issues_url":"https://api.github.com/repos/amplab/ray/issues{/number}","pulls_url":"https://api.github.com/repos/amplab/ray/pulls{/number}","milestones_url":"https://api.github.com/repos/amplab/ray/milestones{/number}","notifications_url":"https://api.github.com/repos/amplab/ray/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/amplab/ray/labels{/name}","releases_url":"https://api.github.com/repos/amplab/ray/releases{/id}","deployments_url":"https://api.github.com/repos/amplab/ray/deployments","created_at":"2016-02-07T22:18:40Z","updated_at":"2016-07-20T01:38:37Z","pushed_at":"2016-07-21T21:32:22Z","git_url":"git://github.com/amplab/ray.git","ssh_url":"git@github.com:amplab/ray.git","clone_url":"https://github.com/amplab/ray.git","svn_url":"https://github.com/amplab/ray","homepage":"","size":915,"stargazers_count":6,"watchers_count":6,"language":"Python","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":2,"mirror_url":null,"open_issues_count":36,"forks":2,"open_issues":36,"watchers":6,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/amplab/ray/pulls/283"},"html":{"href":"https://github.com/amplab/ray/pull/283"},"issue":{"href":"https://api.github.com/repos/amplab/ray/issues/283"},"comments":{"href":"https://api.github.com/repos/amplab/ray/issues/283/comments"},"review_comments":{"href":"https://api.github.com/repos/amplab/ray/pulls/283/comments"},"review_comment":{"href":"https://api.github.com/repos/amplab/ray/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/amplab/ray/pulls/283/commits"},"statuses":{"href":"https://api.github.com/repos/amplab/ray/statuses/901639dfa2df4238f089254203e21d32dec8c636"}}}}
{"actor":{"display_login":"mehrdadn"}}