{"action":"created","issue":{"url":"https://api.github.com/repos/ShangtongZhang/DeepRL/issues/31","repository_url":"https://api.github.com/repos/ShangtongZhang/DeepRL","labels_url":"https://api.github.com/repos/ShangtongZhang/DeepRL/issues/31/labels{/name}","comments_url":"https://api.github.com/repos/ShangtongZhang/DeepRL/issues/31/comments","events_url":"https://api.github.com/repos/ShangtongZhang/DeepRL/issues/31/events","html_url":"https://github.com/ShangtongZhang/DeepRL/issues/31","id":324332060,"number":31,"title":"Option-Critic doesn't seem to converge","user":{"login":"flrndttrch","id":15138061,"avatar_url":"https://avatars2.githubusercontent.com/u/15138061?v=4","gravatar_id":"","url":"https://api.github.com/users/flrndttrch","html_url":"https://github.com/flrndttrch","followers_url":"https://api.github.com/users/flrndttrch/followers","following_url":"https://api.github.com/users/flrndttrch/following{/other_user}","gists_url":"https://api.github.com/users/flrndttrch/gists{/gist_id}","starred_url":"https://api.github.com/users/flrndttrch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/flrndttrch/subscriptions","organizations_url":"https://api.github.com/users/flrndttrch/orgs","repos_url":"https://api.github.com/users/flrndttrch/repos","events_url":"https://api.github.com/users/flrndttrch/events{/privacy}","received_events_url":"https://api.github.com/users/flrndttrch/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-05-18T09:16:54Z","updated_at":"2018-05-18T15:39:55Z","closed_at":"2018-05-18T13:44:08Z","author_association":"NONE","body":"Hi,\r\nI used your implementation of the OC framework with the cartpole environment and it doesn't seem to converge.\r\n\r\nOne questions regarding your implementation in the iteration() method of the OptionCriticAgent class:\r\nWhy do you calculate the returns only for the current option (prev_option)? What if during the rollout the option changed? Shouldn't the option parameters be updated right after the option ended?\r\n\r\nBest regards\r\n"},"comment":{"url":"https://api.github.com/repos/ShangtongZhang/DeepRL/issues/comments/390247277","html_url":"https://github.com/ShangtongZhang/DeepRL/issues/31#issuecomment-390247277","issue_url":"https://api.github.com/repos/ShangtongZhang/DeepRL/issues/31","id":390247277,"user":{"login":"flrndttrch","id":15138061,"avatar_url":"https://avatars2.githubusercontent.com/u/15138061?v=4","gravatar_id":"","url":"https://api.github.com/users/flrndttrch","html_url":"https://github.com/flrndttrch","followers_url":"https://api.github.com/users/flrndttrch/followers","following_url":"https://api.github.com/users/flrndttrch/following{/other_user}","gists_url":"https://api.github.com/users/flrndttrch/gists{/gist_id}","starred_url":"https://api.github.com/users/flrndttrch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/flrndttrch/subscriptions","organizations_url":"https://api.github.com/users/flrndttrch/orgs","repos_url":"https://api.github.com/users/flrndttrch/repos","events_url":"https://api.github.com/users/flrndttrch/events{/privacy}","received_events_url":"https://api.github.com/users/flrndttrch/received_events","type":"User","site_admin":false},"created_at":"2018-05-18T15:39:55Z","updated_at":"2018-05-18T15:39:55Z","author_association":"NONE","body":"After 62275 steps the mean reward is 44.2. The max is 200. Additionally the variance between the values is very high. \r\n\r\nWhy do you think that converges?"}}
{"actor":{"display_login":"flrndttrch"}}