{"action":"closed","issue":{"url":"https://api.github.com/repos/Unity-Technologies/ml-agents/issues/692","repository_url":"https://api.github.com/repos/Unity-Technologies/ml-agents","labels_url":"https://api.github.com/repos/Unity-Technologies/ml-agents/issues/692/labels{/name}","comments_url":"https://api.github.com/repos/Unity-Technologies/ml-agents/issues/692/comments","events_url":"https://api.github.com/repos/Unity-Technologies/ml-agents/issues/692/events","html_url":"https://github.com/Unity-Technologies/ml-agents/issues/692","id":318364927,"node_id":"MDU6SXNzdWUzMTgzNjQ5Mjc=","number":692,"title":"Different behavior between Editor and training","user":{"login":"m-ronchi","id":35332423,"node_id":"MDQ6VXNlcjM1MzMyNDIz","avatar_url":"https://avatars1.githubusercontent.com/u/35332423?v=4","gravatar_id":"","url":"https://api.github.com/users/m-ronchi","html_url":"https://github.com/m-ronchi","followers_url":"https://api.github.com/users/m-ronchi/followers","following_url":"https://api.github.com/users/m-ronchi/following{/other_user}","gists_url":"https://api.github.com/users/m-ronchi/gists{/gist_id}","starred_url":"https://api.github.com/users/m-ronchi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/m-ronchi/subscriptions","organizations_url":"https://api.github.com/users/m-ronchi/orgs","repos_url":"https://api.github.com/users/m-ronchi/repos","events_url":"https://api.github.com/users/m-ronchi/events{/privacy}","received_events_url":"https://api.github.com/users/m-ronchi/received_events","type":"User","site_admin":false},"labels":[{"id":687381587,"node_id":"MDU6TGFiZWw2ODczODE1ODc=","url":"https://api.github.com/repos/Unity-Technologies/ml-agents/labels/help-wanted","name":"help-wanted","color":"33aa3f","default":false}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2018-04-27T10:53:19Z","updated_at":"2018-07-17T19:52:38Z","closed_at":"2018-07-17T19:52:38Z","author_association":"NONE","body":"Hi,\r\nI am building my game for training, and I noticed that behaviour is different between editor and build:\r\nin particular, AcademyReset seems to be called twice, without resetting the agents.\r\n\r\nsetup:\r\nAcademy (subclassed): max steps 50, training: quality level = 0, time scale = 1,  other settings default\r\n- Brain (external in build, internal in editor): vector obs = 244 continuous, visual obs = 0, action = 40 discrete\r\n\r\nAgents (prefab): max step 0, reset on done = true, on demand decisions = true\r\n\r\ngame loop:\r\ngame is a 2-player turn based card game (there is a 4-player variant)\r\n\r\n- AcademyReset:\r\n    - create new game\r\n    - if first time, instantiate one agent per player (GiveBrain to the prefab before instantiating to avoid null brain warning)\r\n    - determine starting player, flag the corresponding agent to act next step\r\n- Agent FixedUpdate:\r\n    - if has the 'your turn' flag, call RequestDecision() and clear the flag\r\n- AgentAction:\r\n    - parse the action and make the move. expecting only the current player will make a move here\r\n    - game logic will:\r\n        - if game is finished, give rewards (+1 to the winner) and call Done on each agent\r\n        - if move is illegal, punish the agent (-10 reward) and call Done on each agent\r\n        - else determine the next player and set the 'your turn' flag\r\n\r\nexpected/observed behavior in editor (untrained model):\r\n1. new game\r\n2. set turn flag\r\n3. agent act\r\n4. abort game (move is likely illegal because model is untrained)\r\n5. goto 1.\r\n\r\nobserved behavior when training:\r\n1. new game\r\n2. set turn flag\r\n3. agent act\r\n4. abort game (move is likely illegal because model is untrained) first game is ok\r\n5. new game\r\n6. set turn flag\r\n7. new game (here, AcademyReset is called before the end of the game)\r\n8. set turn flag\r\n9. both agent act, one of the moves is obviously illegal because it's not their turn\r\n10. abort game twice (and give double punishment)\r\n11. goto 5."}}
{"actor":{"display_login":"eshvk"}}