{"action":"created","issue":{"url":"https://api.github.com/repos/HorizonRobotics/alf/issues/93","repository_url":"https://api.github.com/repos/HorizonRobotics/alf","labels_url":"https://api.github.com/repos/HorizonRobotics/alf/issues/93/labels{/name}","comments_url":"https://api.github.com/repos/HorizonRobotics/alf/issues/93/comments","events_url":"https://api.github.com/repos/HorizonRobotics/alf/issues/93/events","html_url":"https://github.com/HorizonRobotics/alf/pull/93","id":464884661,"node_id":"MDExOlB1bGxSZXF1ZXN0Mjk1MDU5MTgx","number":93,"title":"async training","user":{"login":"hnyu","id":51248379,"node_id":"MDQ6VXNlcjUxMjQ4Mzc5","avatar_url":"https://avatars0.githubusercontent.com/u/51248379?v=4","gravatar_id":"","url":"https://api.github.com/users/hnyu","html_url":"https://github.com/hnyu","followers_url":"https://api.github.com/users/hnyu/followers","following_url":"https://api.github.com/users/hnyu/following{/other_user}","gists_url":"https://api.github.com/users/hnyu/gists{/gist_id}","starred_url":"https://api.github.com/users/hnyu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hnyu/subscriptions","organizations_url":"https://api.github.com/users/hnyu/orgs","repos_url":"https://api.github.com/users/hnyu/repos","events_url":"https://api.github.com/users/hnyu/events{/privacy}","received_events_url":"https://api.github.com/users/hnyu/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2019-07-06T19:41:37Z","updated_at":"2019-07-08T17:12:45Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/HorizonRobotics/alf/pulls/93","html_url":"https://github.com/HorizonRobotics/alf/pull/93","diff_url":"https://github.com/HorizonRobotics/alf/pull/93.diff","patch_url":"https://github.com/HorizonRobotics/alf/pull/93.patch"},"body":"Sorry about pushing such a big change in only one commit. \r\n\r\nAnalysis on async training of PPO humanoid experiments is at \r\nhttps://haonanyu.quip.com/GCVsAb2If1bf/Async-PPO-for-training-humanoid\r\n\r\nTODOs:\r\n1. implement V-Trace for off-policy algorithms\r\n2. merge the driver, trainer, and main files with the current off-policy ones\r\n3. implement our customized experience replay buffer for async training data, mostly for env id alignment when storing a subset of env data into the buffer"},"comment":{"url":"https://api.github.com/repos/HorizonRobotics/alf/issues/comments/509313128","html_url":"https://github.com/HorizonRobotics/alf/pull/93#issuecomment-509313128","issue_url":"https://api.github.com/repos/HorizonRobotics/alf/issues/93","id":509313128,"node_id":"MDEyOklzc3VlQ29tbWVudDUwOTMxMzEyOA==","user":{"login":"hnyu","id":51248379,"node_id":"MDQ6VXNlcjUxMjQ4Mzc5","avatar_url":"https://avatars0.githubusercontent.com/u/51248379?v=4","gravatar_id":"","url":"https://api.github.com/users/hnyu","html_url":"https://github.com/hnyu","followers_url":"https://api.github.com/users/hnyu/followers","following_url":"https://api.github.com/users/hnyu/following{/other_user}","gists_url":"https://api.github.com/users/hnyu/gists{/gist_id}","starred_url":"https://api.github.com/users/hnyu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hnyu/subscriptions","organizations_url":"https://api.github.com/users/hnyu/orgs","repos_url":"https://api.github.com/users/hnyu/repos","events_url":"https://api.github.com/users/hnyu/events{/privacy}","received_events_url":"https://api.github.com/users/hnyu/received_events","type":"User","site_admin":false},"created_at":"2019-07-08T17:12:45Z","updated_at":"2019-07-08T17:12:45Z","author_association":"CONTRIBUTOR","body":"> asynchronous and synchronous distributed training is now supported in TF 2.0 Beta, it seems it's much simpler .\r\n\r\nThis is not about the general deep net training itself. It's about the asynchronization between environment rollout and parameter update."}}
{"actor":{"display_login":"hnyu"}}