{"action":"created","issue":{"url":"https://api.github.com/repos/Unity-Technologies/ml-agents/issues/1083","repository_url":"https://api.github.com/repos/Unity-Technologies/ml-agents","labels_url":"https://api.github.com/repos/Unity-Technologies/ml-agents/issues/1083/labels{/name}","comments_url":"https://api.github.com/repos/Unity-Technologies/ml-agents/issues/1083/comments","events_url":"https://api.github.com/repos/Unity-Technologies/ml-agents/issues/1083/events","html_url":"https://github.com/Unity-Technologies/ml-agents/pull/1083","id":349682171,"node_id":"MDExOlB1bGxSZXF1ZXN0MjA3NzQxMzgy","number":1083,"title":"ML-Agents Code Restructure","user":{"login":"dericp","id":13430946,"node_id":"MDQ6VXNlcjEzNDMwOTQ2","avatar_url":"https://avatars0.githubusercontent.com/u/13430946?v=4","gravatar_id":"","url":"https://api.github.com/users/dericp","html_url":"https://github.com/dericp","followers_url":"https://api.github.com/users/dericp/followers","following_url":"https://api.github.com/users/dericp/following{/other_user}","gists_url":"https://api.github.com/users/dericp/gists{/gist_id}","starred_url":"https://api.github.com/users/dericp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dericp/subscriptions","organizations_url":"https://api.github.com/users/dericp/orgs","repos_url":"https://api.github.com/users/dericp/repos","events_url":"https://api.github.com/users/dericp/events{/privacy}","received_events_url":"https://api.github.com/users/dericp/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2018-08-10T23:13:30Z","updated_at":"2018-08-23T21:52:27Z","closed_at":null,"author_association":"COLLABORATOR","pull_request":{"url":"https://api.github.com/repos/Unity-Technologies/ml-agents/pulls/1083","html_url":"https://github.com/Unity-Technologies/ml-agents/pull/1083","diff_url":"https://github.com/Unity-Technologies/ml-agents/pull/1083.diff","patch_url":"https://github.com/Unity-Technologies/ml-agents/pull/1083.patch"},"body":"## Summary\r\n\r\nThe Python code in ML-Agents suffers from a package hierarchy that makes testing and publishing to PIP difficult. I propose that moving the `unityagents` and `unitytrainers` packages into a single top level `mlagents` python package will make the separation of these packages more clear, allow us to decouple their tests, and make it possible to publish to pip easily.\r\n\r\n## Background\r\n\r\nWhile working on moving curriculum learning out of unityagents and into unitytrainers, I wanted to be able to make a test hierarchy. Specifically, I wanted to create two separate test directories tests/unityagents and tests/unitytrainers.\r\n\r\nBecause of the way the Python code is currently packaged, this was difficult to do. Imports broke, and eventually the only way to get any tests to pass was to run `python -m pytest tests/` which is less than ideal.\r\n\r\nInitially, we concluded that it would probably be a better use of my time to work on something else. However, with Arthur adding a gym environment to the repo, it seems that it might be better to perform a restructure as soon as possible.\r\n\r\n## Goals\r\n\r\n- Run all python tests from top level `ml-agents` directory by just running `$ pytest`.\r\n- The `tests` package should mirror the structure of the Python code.\r\n- Name the Python packages more intuitively.\r\n- Publish `mlagents` package to PyPi.\r\n\r\n## Plan\r\n\r\n- Move directories around according to the new packaging scheme.\r\n- Fix imports and tests.\r\n- Expose learn.py as an environment script.\r\n- Update documentation to reflect the new package structure.\r\n- Update Docker container to reflect the new package structure.\r\n- Upload `mlagents` to TestPyPi.\r\n- Upload `mlagents` to PyPi.\r\n\r\n## Things done that were not in original plan\r\n\r\n- Added ml-agents-protobuf to the ml-agents repo. Documentation has been improved.\r\n- Documentation that I had to edit has been changed to be compliant with markdownlint.\r\n\r\n## Notes\r\n\r\nThere are a few work-items I could use help with:\r\n- Verify that the Docker container is working properly.\r\n  - I have been able to build and train an agent in the new Docker container but it would be great if @eshvk could sanity check my work.\r\n- Verify that the `gym-unity` wrapper still works properly. @awjuliani \r\n- Verify that the \"Training on AWS\" documentation is correct.\r\n- Verify that all behavior has remained identical to before.\r\n  - All unit tests are passing but this does not necessarily give me confidence that all the code is working as it should. I guess we could either write more unit tests or empirically verify the code's correctness.\r\n- Verify that all documentation is aligned with the new structure.\r\n\r\n## Tips for Reviewing\r\n\r\nThis is a huge PR mostly because of edits in the documentation. I would suggest reviewing one commit at a time since each commit is a logical chunk of work.\r\n\r\nTo install `mlagents` from TestPyPi, run\r\n```\r\npip install --index-url https://test.pypi.org/simple/ mlagents\r\n```"},"comment":{"url":"https://api.github.com/repos/Unity-Technologies/ml-agents/issues/comments/415583106","html_url":"https://github.com/Unity-Technologies/ml-agents/pull/1083#issuecomment-415583106","issue_url":"https://api.github.com/repos/Unity-Technologies/ml-agents/issues/1083","id":415583106,"node_id":"MDEyOklzc3VlQ29tbWVudDQxNTU4MzEwNg==","user":{"login":"awjuliani","id":9065325,"node_id":"MDQ6VXNlcjkwNjUzMjU=","avatar_url":"https://avatars2.githubusercontent.com/u/9065325?v=4","gravatar_id":"","url":"https://api.github.com/users/awjuliani","html_url":"https://github.com/awjuliani","followers_url":"https://api.github.com/users/awjuliani/followers","following_url":"https://api.github.com/users/awjuliani/following{/other_user}","gists_url":"https://api.github.com/users/awjuliani/gists{/gist_id}","starred_url":"https://api.github.com/users/awjuliani/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/awjuliani/subscriptions","organizations_url":"https://api.github.com/users/awjuliani/orgs","repos_url":"https://api.github.com/users/awjuliani/repos","events_url":"https://api.github.com/users/awjuliani/events{/privacy}","received_events_url":"https://api.github.com/users/awjuliani/received_events","type":"User","site_admin":false},"created_at":"2018-08-23T21:52:27Z","updated_at":"2018-08-23T21:52:27Z","author_association":"COLLABORATOR","body":"Another question is whether we want to split up the C# plugin from the environments here as well.\r\n@vincentpierre did you test the symlinks on Windows?"}}
{"actor":{"display_login":"awjuliani"}}