{"action":"opened","issue":{"url":"https://api.github.com/repos/Unity-Technologies/ml-agents/issues/310","repository_url":"https://api.github.com/repos/Unity-Technologies/ml-agents","labels_url":"https://api.github.com/repos/Unity-Technologies/ml-agents/issues/310/labels{/name}","comments_url":"https://api.github.com/repos/Unity-Technologies/ml-agents/issues/310/comments","events_url":"https://api.github.com/repos/Unity-Technologies/ml-agents/issues/310/events","html_url":"https://github.com/Unity-Technologies/ml-agents/issues/310","id":294080605,"number":310,"title":"Training freezes at the bad result and not improving","user":{"login":"cxovrika","id":29161063,"avatar_url":"https://avatars1.githubusercontent.com/u/29161063?v=4","gravatar_id":"","url":"https://api.github.com/users/cxovrika","html_url":"https://github.com/cxovrika","followers_url":"https://api.github.com/users/cxovrika/followers","following_url":"https://api.github.com/users/cxovrika/following{/other_user}","gists_url":"https://api.github.com/users/cxovrika/gists{/gist_id}","starred_url":"https://api.github.com/users/cxovrika/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cxovrika/subscriptions","organizations_url":"https://api.github.com/users/cxovrika/orgs","repos_url":"https://api.github.com/users/cxovrika/repos","events_url":"https://api.github.com/users/cxovrika/events{/privacy}","received_events_url":"https://api.github.com/users/cxovrika/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-02-03T03:42:25Z","updated_at":"2018-02-03T03:42:25Z","closed_at":null,"author_association":"NONE","body":"Hello!\r\n\r\nI am trying to train agent on a simple game: you are a capsule on a plane, you can move(WASD) and shoot upwards, dummy enemy is patrolling from left to the right of the board, and is shooting downwards every 2 sec.\r\n\r\nGIF showing the game:  http://g.recordit.co/jJlwCkLni5.gif\r\n\r\nwhile training, after every agentStep i give -10 reward for dying of falling off the platform, 20 reward for killing the enemy, -0.01 reward for trying to shoot while it is on cooldown and -0.005 otherwise.\r\n\r\nthe training goes really bad, as you can see from here: https://pastebin.com/t0zZxzJA\r\nfor some reason it goes from random moves(which give nearly 2.5 mean reward) to the moves that guarantee to give the -10 reward (STD of reward drops to almost zero as you can see)\r\n\r\nwhen i gave the given bad model to the agent, for some reason that i don't understand it was moving only on the diagonal, from it's starting position:  http://g.recordit.co/LSXcEcSk47.gif\r\n\r\nsome additional info\r\nhyperparameters: https://pastebin.com/iir3R4sH\r\ninfo about brain: https://snag.gy/4lW0oN.jpg\r\nstate stores information about your, enemy's and enemy's projectile's position and velocity\r\n\r\nany ideas why is this happening?\r\n\r\nthanks in advance for any help!\r\n\r\n"}}
{"actor":{"display_login":"cxovrika"}}