{"action":"created","issue":{"url":"https://api.github.com/repos/Kaixhin/Atari/issues/48","repository_url":"https://api.github.com/repos/Kaixhin/Atari","labels_url":"https://api.github.com/repos/Kaixhin/Atari/issues/48/labels{/name}","comments_url":"https://api.github.com/repos/Kaixhin/Atari/issues/48/comments","events_url":"https://api.github.com/repos/Kaixhin/Atari/issues/48/events","html_url":"https://github.com/Kaixhin/Atari/pull/48","id":163731714,"number":48,"title":"Correct entropy regularisation","user":{"login":"Kaixhin","id":991891,"avatar_url":"https://avatars.githubusercontent.com/u/991891?v=3","gravatar_id":"","url":"https://api.github.com/users/Kaixhin","html_url":"https://github.com/Kaixhin","followers_url":"https://api.github.com/users/Kaixhin/followers","following_url":"https://api.github.com/users/Kaixhin/following{/other_user}","gists_url":"https://api.github.com/users/Kaixhin/gists{/gist_id}","starred_url":"https://api.github.com/users/Kaixhin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Kaixhin/subscriptions","organizations_url":"https://api.github.com/users/Kaixhin/orgs","repos_url":"https://api.github.com/users/Kaixhin/repos","events_url":"https://api.github.com/users/Kaixhin/events{/privacy}","received_events_url":"https://api.github.com/users/Kaixhin/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2016-07-04T18:55:09Z","updated_at":"2016-07-12T11:14:13Z","closed_at":"2016-07-12T11:14:13Z","pull_request":{"url":"https://api.github.com/repos/Kaixhin/Atari/pulls/48","html_url":"https://github.com/Kaixhin/Atari/pull/48","diff_url":"https://github.com/Kaixhin/Atari/pull/48.diff","patch_url":"https://github.com/Kaixhin/Atari/pull/48.patch"},"body":"**DO NOT MERGE YET**\r\n\r\n@lake4790k I talked to someone about the entropy regularisation and they worked through the derivatives by hand and concluded that gradient for each action is independent, and therefore the sum shouldn't be used. I've tested with `demo-async-a3c` and it doesn't work with the current value of `beta`, but does learn reasonably when `beta` is reduced. The learning curve is more shallow/the agent takes longer to learn, presumably because of the increased exploration.\r\n\r\nIt's quite likely that this only helps with more complicated games where the policy is not straightforward. I'm going to try a run with Beam Rider using this and then with your solution to see what works, but we might have to ask the authors about this one."},"comment":{"url":"https://api.github.com/repos/Kaixhin/Atari/issues/comments/232010507","html_url":"https://github.com/Kaixhin/Atari/pull/48#issuecomment-232010507","issue_url":"https://api.github.com/repos/Kaixhin/Atari/issues/48","id":232010507,"user":{"login":"Kaixhin","id":991891,"avatar_url":"https://avatars.githubusercontent.com/u/991891?v=3","gravatar_id":"","url":"https://api.github.com/users/Kaixhin","html_url":"https://github.com/Kaixhin","followers_url":"https://api.github.com/users/Kaixhin/followers","following_url":"https://api.github.com/users/Kaixhin/following{/other_user}","gists_url":"https://api.github.com/users/Kaixhin/gists{/gist_id}","starred_url":"https://api.github.com/users/Kaixhin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Kaixhin/subscriptions","organizations_url":"https://api.github.com/users/Kaixhin/orgs","repos_url":"https://api.github.com/users/Kaixhin/repos","events_url":"https://api.github.com/users/Kaixhin/events{/privacy}","received_events_url":"https://api.github.com/users/Kaixhin/received_events","type":"User","site_admin":false},"created_at":"2016-07-12T11:14:13Z","updated_at":"2016-07-12T11:14:13Z","body":"I've run several experiments and don't have any good results at all, so I'm closing this. I'm not 100% sure that your calculation is correct but at least it works."}}
{"actor":{"display_login":"Kaixhin"}}