{"action":"created","issue":{"url":"https://api.github.com/repos/ray-project/ray/issues/1856","repository_url":"https://api.github.com/repos/ray-project/ray","labels_url":"https://api.github.com/repos/ray-project/ray/issues/1856/labels{/name}","comments_url":"https://api.github.com/repos/ray-project/ray/issues/1856/comments","events_url":"https://api.github.com/repos/ray-project/ray/issues/1856/events","html_url":"https://github.com/ray-project/ray/issues/1856","id":312371560,"node_id":"MDU6SXNzdWUzMTIzNzE1NjA=","number":1856,"title":"Getting pytorch model weights on cuda device gives no dimension tensors","user":{"login":"JunhongXu","id":13001487,"node_id":"MDQ6VXNlcjEzMDAxNDg3","avatar_url":"https://avatars2.githubusercontent.com/u/13001487?v=4","gravatar_id":"","url":"https://api.github.com/users/JunhongXu","html_url":"https://github.com/JunhongXu","followers_url":"https://api.github.com/users/JunhongXu/followers","following_url":"https://api.github.com/users/JunhongXu/following{/other_user}","gists_url":"https://api.github.com/users/JunhongXu/gists{/gist_id}","starred_url":"https://api.github.com/users/JunhongXu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JunhongXu/subscriptions","organizations_url":"https://api.github.com/users/JunhongXu/orgs","repos_url":"https://api.github.com/users/JunhongXu/repos","events_url":"https://api.github.com/users/JunhongXu/events{/privacy}","received_events_url":"https://api.github.com/users/JunhongXu/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"richardliaw","id":4529381,"node_id":"MDQ6VXNlcjQ1MjkzODE=","avatar_url":"https://avatars0.githubusercontent.com/u/4529381?v=4","gravatar_id":"","url":"https://api.github.com/users/richardliaw","html_url":"https://github.com/richardliaw","followers_url":"https://api.github.com/users/richardliaw/followers","following_url":"https://api.github.com/users/richardliaw/following{/other_user}","gists_url":"https://api.github.com/users/richardliaw/gists{/gist_id}","starred_url":"https://api.github.com/users/richardliaw/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/richardliaw/subscriptions","organizations_url":"https://api.github.com/users/richardliaw/orgs","repos_url":"https://api.github.com/users/richardliaw/repos","events_url":"https://api.github.com/users/richardliaw/events{/privacy}","received_events_url":"https://api.github.com/users/richardliaw/received_events","type":"User","site_admin":false},"assignees":[{"login":"richardliaw","id":4529381,"node_id":"MDQ6VXNlcjQ1MjkzODE=","avatar_url":"https://avatars0.githubusercontent.com/u/4529381?v=4","gravatar_id":"","url":"https://api.github.com/users/richardliaw","html_url":"https://github.com/richardliaw","followers_url":"https://api.github.com/users/richardliaw/followers","following_url":"https://api.github.com/users/richardliaw/following{/other_user}","gists_url":"https://api.github.com/users/richardliaw/gists{/gist_id}","starred_url":"https://api.github.com/users/richardliaw/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/richardliaw/subscriptions","organizations_url":"https://api.github.com/users/richardliaw/orgs","repos_url":"https://api.github.com/users/richardliaw/repos","events_url":"https://api.github.com/users/richardliaw/events{/privacy}","received_events_url":"https://api.github.com/users/richardliaw/received_events","type":"User","site_admin":false}],"milestone":null,"comments":8,"created_at":"2018-04-09T02:31:00Z","updated_at":"2018-10-28T01:04:48Z","closed_at":null,"author_association":"NONE","body":"<!--\r\nGeneral questions should be asked on the mailing list ray-dev@googlegroups.com.\r\n\r\nBefore submitting an issue, please fill out the following form.\r\n-->\r\n\r\n### System information\r\n- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**:\r\nLinux Ubuntu 16.04\r\n- **Ray installed from (source or binary)**:\r\nBinary\r\n- **Ray version**:\r\n0.4.0\r\n- **Python version**:\r\n3.6\r\n- **Exact command to reproduce**:\r\n\r\n<!--\r\nYou can obtain the Ray version with\r\n\r\npython -c \"import ray; print(ray.__version__)\"\r\n-->\r\n\r\n### Describe the problem\r\n<!-- Describe the problem clearly here. -->\r\n\r\n\r\nI am trying to write a pytorch version of PPO in Ray. When I implement ```get_weights() and set_weights()``` method, I got this warning and the weight tensors I got from remote does not have any element (The problem only exists when the tensors are on GPUs.):\r\n\r\n> WARNING: Serializing objects of type <class 'torch.cuda.FloatTensor'> by expanding them as dictionaries of their fields. This behavior may be incorrect in some cases.\r\n \r\n\r\n\r\n### Source code / logs\r\n<!-- Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached. Try to provide a reproducible test case that is the bare minimum necessary to generate the problem. -->\r\n\r\nThe code I used: \r\n\r\n```\r\n\r\nclass TorchPPOAgent(Agent):\r\n\r\n    def _init(self):\r\n        self.local_evaluator = TorchPPOEvaluator(env_creator=self.env_creator, config=self.config)\r\n\r\n        RemoteEvaluators = ray.remote(**self.config['worker_resources'])(TorchPPOEvaluator)\r\n        self.remote_evaluators = []\r\n        for _ in range(self.config['num_workers']):\r\n            self.remote_evaluators.append(RemoteEvaluators.remote(self.config, self.env_creator))\r\n\t\t# other initialization stuff ...\r\n\t\r\n\t...\r\n\r\n    def _train(self):\r\n        local_model = self.local_evaluator\r\n        remote_models = self.remote_evaluators\r\n\r\n        # sync parameters\r\n        weight_id = ray.put(local_model.get_weights())\r\n\r\n \t\t# the tensors in this state dict are torch.cuda.FloatTensor with no dimensions\r\n\t\tprint(ray.get(weight_id))\t\r\n\r\n\r\n\t\r\nclass TorchPPOEvaluator(PolicyEvaluator):\r\n\r\n    def __init__(self, config, env_creator):\r\n\t\t# ... initialization stuff ...\r\n\t\tif self.config['cuda']:\r\n        \tself.net.cuda()\r\n\t\r\n\t# set weights and get weights function\r\n    def set_weights(self, weights):\r\n\t\t# if I first transfer the GPU tensors to CPU, it works fine.\r\n        # if self.config['cuda']:\r\n        #     for weight in weights:\r\n        #         weights[weight] = weights[weight].cuda()\r\n        self.net.load_state_dict(weights)\r\n\r\n    def get_weights(self):\r\n        state_dict = self.net.state_dict()\r\n\t\t# if I first transfer the GPU tensors to CPU, it works fine.\r\n        # if self.config['cuda']:\r\n        #     for key in state_dict:\r\n        #         state_dict[key] = state_dict[key].cpu()\r\n        return state_dict\r\n\r\n```\r\n"},"comment":{"url":"https://api.github.com/repos/ray-project/ray/issues/comments/433667809","html_url":"https://github.com/ray-project/ray/issues/1856#issuecomment-433667809","issue_url":"https://api.github.com/repos/ray-project/ray/issues/1856","id":433667809,"node_id":"MDEyOklzc3VlQ29tbWVudDQzMzY2NzgwOQ==","user":{"login":"valiantljk","id":1396867,"node_id":"MDQ6VXNlcjEzOTY4Njc=","avatar_url":"https://avatars2.githubusercontent.com/u/1396867?v=4","gravatar_id":"","url":"https://api.github.com/users/valiantljk","html_url":"https://github.com/valiantljk","followers_url":"https://api.github.com/users/valiantljk/followers","following_url":"https://api.github.com/users/valiantljk/following{/other_user}","gists_url":"https://api.github.com/users/valiantljk/gists{/gist_id}","starred_url":"https://api.github.com/users/valiantljk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/valiantljk/subscriptions","organizations_url":"https://api.github.com/users/valiantljk/orgs","repos_url":"https://api.github.com/users/valiantljk/repos","events_url":"https://api.github.com/users/valiantljk/events{/privacy}","received_events_url":"https://api.github.com/users/valiantljk/received_events","type":"User","site_admin":false},"created_at":"2018-10-28T01:04:48Z","updated_at":"2018-10-28T01:04:48Z","author_association":"NONE","body":"I have arbitrary python class, it looks like using Plasma, it requires me to write all the custom serializer and deserializer for those classes, is there any easier way?"}}
{"actor":{"display_login":"valiantljk"}}