{"action":"created","issue":{"url":"https://api.github.com/repos/dask/distributed/issues/1820","repository_url":"https://api.github.com/repos/dask/distributed","labels_url":"https://api.github.com/repos/dask/distributed/issues/1820/labels{/name}","comments_url":"https://api.github.com/repos/dask/distributed/issues/1820/comments","events_url":"https://api.github.com/repos/dask/distributed/issues/1820/events","html_url":"https://github.com/dask/distributed/issues/1820","id":303215704,"number":1820,"title":"Client returned by distributed.get_client() on LocalCluster worker has unknown properties","user":{"login":"dancollins34","id":33334385,"avatar_url":"https://avatars1.githubusercontent.com/u/33334385?v=4","gravatar_id":"","url":"https://api.github.com/users/dancollins34","html_url":"https://github.com/dancollins34","followers_url":"https://api.github.com/users/dancollins34/followers","following_url":"https://api.github.com/users/dancollins34/following{/other_user}","gists_url":"https://api.github.com/users/dancollins34/gists{/gist_id}","starred_url":"https://api.github.com/users/dancollins34/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dancollins34/subscriptions","organizations_url":"https://api.github.com/users/dancollins34/orgs","repos_url":"https://api.github.com/users/dancollins34/repos","events_url":"https://api.github.com/users/dancollins34/events{/privacy}","received_events_url":"https://api.github.com/users/dancollins34/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2018-03-07T18:47:21Z","updated_at":"2018-03-07T19:48:27Z","closed_at":null,"author_association":"NONE","body":"The client returned by distributed.get_client() on a LocalCluster worker appears to be the same client used by the original client which created the cluster, as opposed to a standard synchronous client.   For example, if you create a local cluster using AioClient() or Client(asynchronous=True), it appears as if the result of distributed.get_client() within a synchronous function called in a task will be a reference to that object.  This is a problem because, as you are now in a synchronous context in a running event loop, it becomes hard to write code which could be run on both a LocalCluster and a connection to a remote cluster.  Is this intended behavior?  As a workaround, I'm calling the following to ensure that the client is always the same, although one would expect this to have some overhead:\r\n\r\n\r\n```python\r\nfrom distributed import get_client, Client\r\n\r\ndef f():\r\n    existing = get_client()  # This may be an AioClient, or a Client() or a Client(asynchronous=True)\r\n    client = Client(address=existing.scheduler.address)\r\n\r\n    # do things with client\r\n\r\n    client.close()\r\n```"},"comment":{"url":"https://api.github.com/repos/dask/distributed/issues/comments/371261807","html_url":"https://github.com/dask/distributed/issues/1820#issuecomment-371261807","issue_url":"https://api.github.com/repos/dask/distributed/issues/1820","id":371261807,"user":{"login":"mrocklin","id":306380,"avatar_url":"https://avatars3.githubusercontent.com/u/306380?v=4","gravatar_id":"","url":"https://api.github.com/users/mrocklin","html_url":"https://github.com/mrocklin","followers_url":"https://api.github.com/users/mrocklin/followers","following_url":"https://api.github.com/users/mrocklin/following{/other_user}","gists_url":"https://api.github.com/users/mrocklin/gists{/gist_id}","starred_url":"https://api.github.com/users/mrocklin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mrocklin/subscriptions","organizations_url":"https://api.github.com/users/mrocklin/orgs","repos_url":"https://api.github.com/users/mrocklin/repos","events_url":"https://api.github.com/users/mrocklin/events{/privacy}","received_events_url":"https://api.github.com/users/mrocklin/received_events","type":"User","site_admin":false},"created_at":"2018-03-07T19:48:27Z","updated_at":"2018-03-07T19:48:27Z","author_association":"OWNER","body":"Yeah, so this works fine when using Tornado.  I haven't personally used the AioClient that much.  \r\n\r\n```python\r\nIn [1]: from dask.distributed import get_client, Client, secede, rejoin\r\n\r\nIn [2]: def generate_graph():\r\n   ...:     # mockup\r\n   ...:     return {\"a\": 3}\r\n   ...: \r\n   ...: def f():\r\n   ...:     existing = get_client() # f was run in an AioClient() so this is an instance of AioClient()\r\n   ...:     futures = []\r\n   ...:     for _ in range(5):\r\n   ...:         future = existing.get(generate_graph(), \"a\", sync=False)  # <class '_asyncio.Task'>\r\n   ...:         futures.append(future)\r\n   ...:     secede()\r\n   ...:     results = existing.gather(futures)  # <class '_asyncio.Future'>\r\n   ...:     rejoin()\r\n   ...:     return results # <class '_asyncio.Future'>\r\n   ...: \r\n\r\nIn [3]: client = Client()\r\n\r\nIn [4]: client.submit(f).result()\r\nOut[4]: [3, 3, 3, 3, 3]\r\n```\r\n\r\ncc @kszucs, who might be able to resolve this.\r\n\r\nIt might also be worth trying this under tornado 5.0 (released yesterday) where the two event loops are identical."}}
{"actor":{"display_login":"mrocklin"}}