{"action":"opened","issue":{"url":"https://api.github.com/repos/keras-rl/keras-rl/issues/202","repository_url":"https://api.github.com/repos/keras-rl/keras-rl","labels_url":"https://api.github.com/repos/keras-rl/keras-rl/issues/202/labels{/name}","comments_url":"https://api.github.com/repos/keras-rl/keras-rl/issues/202/comments","events_url":"https://api.github.com/repos/keras-rl/keras-rl/issues/202/events","html_url":"https://github.com/keras-rl/keras-rl/issues/202","id":319907062,"number":202,"title":"Strange training results?","user":{"login":"olavt","id":3481146,"avatar_url":"https://avatars0.githubusercontent.com/u/3481146?v=4","gravatar_id":"","url":"https://api.github.com/users/olavt","html_url":"https://github.com/olavt","followers_url":"https://api.github.com/users/olavt/followers","following_url":"https://api.github.com/users/olavt/following{/other_user}","gists_url":"https://api.github.com/users/olavt/gists{/gist_id}","starred_url":"https://api.github.com/users/olavt/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/olavt/subscriptions","organizations_url":"https://api.github.com/users/olavt/orgs","repos_url":"https://api.github.com/users/olavt/repos","events_url":"https://api.github.com/users/olavt/events{/privacy}","received_events_url":"https://api.github.com/users/olavt/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-05-03T12:50:56Z","updated_at":"2018-05-03T12:50:56Z","closed_at":null,"author_association":"NONE","body":"I have created a very simple gym environment and some test code on top of keras-rl. The simple gym environment is representing a ventilation fan with 4 speeds. The reward is extremely simple based on fan speed, the higher speed equals less reward. Optimal solution is to keep the fan speed at 0.\r\n\r\nThe problem is that both training and testing of the model experiences some very \"bad\" results when it comes to achieving a reasonably high reward as shown in this chart:\r\n\r\n![rlperformance](https://user-images.githubusercontent.com/3481146/39576873-ed88da28-4edf-11e8-8a3e-d07d47a62c7c.JPG)\r\n\r\nThe chart is based on running 1 episode (with 60 steps) of training, then 1 episode (60 steps) of test. Then it's repeated in a loop. \r\n\r\nThe chart shows the returned reward for returned by \"fit\" (training) and \"test\" for each iteration in the loop. The chart shows that there are severe drops in performance of the agent and it does not seem to improve over time either. It looks like it gets trained reasonably well over the first 20 iterations, but after that it behaves a bit \"randomly\" both when it comes to training and testing. I'm using eps=0.01, so I wouldn't think that it should behave this bad.\r\n\r\nMy goal would be to get it to perform a lot more consistent. I expect some dips due to exploration, but the chart results looks like it's not doing well.\r\n\r\nWhat can be the reasons for the behavior seen?\r\n\r\nThe complete code can be found here:\r\n\r\nhttps://github.com/olavt/gym_co2_ventilation (example file used is test_keras_rl_continious.py)\r\n"}}
{"actor":{"display_login":"olavt"}}