{"action":"created","issue":{"url":"https://api.github.com/repos/tensorflow/agents/issues/42","repository_url":"https://api.github.com/repos/tensorflow/agents","labels_url":"https://api.github.com/repos/tensorflow/agents/issues/42/labels{/name}","comments_url":"https://api.github.com/repos/tensorflow/agents/issues/42/comments","events_url":"https://api.github.com/repos/tensorflow/agents/issues/42/events","html_url":"https://github.com/tensorflow/agents/issues/42","id":422503586,"node_id":"MDU6SXNzdWU0MjI1MDM1ODY=","number":42,"title":"Rainbow agent implementation?","user":{"login":"wszjzhang","id":7960688,"node_id":"MDQ6VXNlcjc5NjA2ODg=","avatar_url":"https://avatars2.githubusercontent.com/u/7960688?v=4","gravatar_id":"","url":"https://api.github.com/users/wszjzhang","html_url":"https://github.com/wszjzhang","followers_url":"https://api.github.com/users/wszjzhang/followers","following_url":"https://api.github.com/users/wszjzhang/following{/other_user}","gists_url":"https://api.github.com/users/wszjzhang/gists{/gist_id}","starred_url":"https://api.github.com/users/wszjzhang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wszjzhang/subscriptions","organizations_url":"https://api.github.com/users/wszjzhang/orgs","repos_url":"https://api.github.com/users/wszjzhang/repos","events_url":"https://api.github.com/users/wszjzhang/events{/privacy}","received_events_url":"https://api.github.com/users/wszjzhang/received_events","type":"User","site_admin":false},"labels":[{"id":1130523733,"node_id":"MDU6TGFiZWwxMTMwNTIzNzMz","url":"https://api.github.com/repos/tensorflow/agents/labels/enhancement","name":"enhancement","color":"a2eeef","default":true}],"state":"closed","locked":false,"assignee":{"login":"sguada","id":1766524,"node_id":"MDQ6VXNlcjE3NjY1MjQ=","avatar_url":"https://avatars3.githubusercontent.com/u/1766524?v=4","gravatar_id":"","url":"https://api.github.com/users/sguada","html_url":"https://github.com/sguada","followers_url":"https://api.github.com/users/sguada/followers","following_url":"https://api.github.com/users/sguada/following{/other_user}","gists_url":"https://api.github.com/users/sguada/gists{/gist_id}","starred_url":"https://api.github.com/users/sguada/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sguada/subscriptions","organizations_url":"https://api.github.com/users/sguada/orgs","repos_url":"https://api.github.com/users/sguada/repos","events_url":"https://api.github.com/users/sguada/events{/privacy}","received_events_url":"https://api.github.com/users/sguada/received_events","type":"User","site_admin":false},"assignees":[{"login":"sguada","id":1766524,"node_id":"MDQ6VXNlcjE3NjY1MjQ=","avatar_url":"https://avatars3.githubusercontent.com/u/1766524?v=4","gravatar_id":"","url":"https://api.github.com/users/sguada","html_url":"https://github.com/sguada","followers_url":"https://api.github.com/users/sguada/followers","following_url":"https://api.github.com/users/sguada/following{/other_user}","gists_url":"https://api.github.com/users/sguada/gists{/gist_id}","starred_url":"https://api.github.com/users/sguada/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sguada/subscriptions","organizations_url":"https://api.github.com/users/sguada/orgs","repos_url":"https://api.github.com/users/sguada/repos","events_url":"https://api.github.com/users/sguada/events{/privacy}","received_events_url":"https://api.github.com/users/sguada/received_events","type":"User","site_admin":false}],"milestone":null,"comments":4,"created_at":"2019-03-19T01:33:26Z","updated_at":"2019-09-06T17:58:27Z","closed_at":"2019-09-06T17:58:27Z","author_association":"NONE","body":"Along the way to enhance DQN, looks like DDQN is also getting implemented. How about the Rainbow algorithm?"},"comment":{"url":"https://api.github.com/repos/tensorflow/agents/issues/comments/528953735","html_url":"https://github.com/tensorflow/agents/issues/42#issuecomment-528953735","issue_url":"https://api.github.com/repos/tensorflow/agents/issues/42","id":528953735,"node_id":"MDEyOklzc3VlQ29tbWVudDUyODk1MzczNQ==","user":{"login":"nealwu","id":726075,"node_id":"MDQ6VXNlcjcyNjA3NQ==","avatar_url":"https://avatars3.githubusercontent.com/u/726075?v=4","gravatar_id":"","url":"https://api.github.com/users/nealwu","html_url":"https://github.com/nealwu","followers_url":"https://api.github.com/users/nealwu/followers","following_url":"https://api.github.com/users/nealwu/following{/other_user}","gists_url":"https://api.github.com/users/nealwu/gists{/gist_id}","starred_url":"https://api.github.com/users/nealwu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nealwu/subscriptions","organizations_url":"https://api.github.com/users/nealwu/orgs","repos_url":"https://api.github.com/users/nealwu/repos","events_url":"https://api.github.com/users/nealwu/events{/privacy}","received_events_url":"https://api.github.com/users/nealwu/received_events","type":"User","site_admin":false},"created_at":"2019-09-06T17:58:27Z","updated_at":"2019-09-06T17:58:27Z","author_association":"MEMBER","body":"Hi all, in case you missed it we currently have Categorical DQN (C51) available in our repo along with n-step updates: https://github.com/tensorflow/agents/tree/master/tf_agents/agents/categorical_dqn\r\n\r\nI also implemented prioritized replay, often considered the third key component of the Rainbow agent. However we actually found that our agent performed just as well and trained more stably without prioritized replay, so we decided to exclude it from our Rainbow implementation.\r\n\r\nOur combined C51 + n-step update agent performs just as well as other Rainbow agents on many of the Atari environments we've tested, so I'm happy to say we can close this issue."}}
{"actor":{"display_login":"nealwu"}}