{"action":"created","issue":{"url":"https://api.github.com/repos/OpenNMT/OpenNMT-py/issues/762","repository_url":"https://api.github.com/repos/OpenNMT/OpenNMT-py","labels_url":"https://api.github.com/repos/OpenNMT/OpenNMT-py/issues/762/labels{/name}","comments_url":"https://api.github.com/repos/OpenNMT/OpenNMT-py/issues/762/comments","events_url":"https://api.github.com/repos/OpenNMT/OpenNMT-py/issues/762/events","html_url":"https://github.com/OpenNMT/OpenNMT-py/pull/762","id":331599942,"node_id":"MDExOlB1bGxSZXF1ZXN0MTk0Mjg5Mzc3","number":762,"title":"[WIP] Code refactoring + Torch distributed support + epoch to step change","user":{"login":"vince62s","id":15141326,"node_id":"MDQ6VXNlcjE1MTQxMzI2","avatar_url":"https://avatars3.githubusercontent.com/u/15141326?v=4","gravatar_id":"","url":"https://api.github.com/users/vince62s","html_url":"https://github.com/vince62s","followers_url":"https://api.github.com/users/vince62s/followers","following_url":"https://api.github.com/users/vince62s/following{/other_user}","gists_url":"https://api.github.com/users/vince62s/gists{/gist_id}","starred_url":"https://api.github.com/users/vince62s/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vince62s/subscriptions","organizations_url":"https://api.github.com/users/vince62s/orgs","repos_url":"https://api.github.com/users/vince62s/repos","events_url":"https://api.github.com/users/vince62s/events{/privacy}","received_events_url":"https://api.github.com/users/vince62s/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2018-06-12T13:58:07Z","updated_at":"2018-06-22T08:09:16Z","closed_at":null,"author_association":"NONE","pull_request":{"url":"https://api.github.com/repos/OpenNMT/OpenNMT-py/pulls/762","html_url":"https://github.com/OpenNMT/OpenNMT-py/pull/762","diff_url":"https://github.com/OpenNMT/OpenNMT-py/pull/762.diff","patch_url":"https://github.com/OpenNMT/OpenNMT-py/pull/762.patch"},"body":"This PR is a work in progress:\r\n\r\nOld models will work as long as they are \"released\". Otherwise we provide a script to upgrade the existing models. The only change comes from the fact that the Optim class stores the path in the checkpoint.\r\n\r\nModules have been dispatched in a way to make the code more readable. If this is not the case, let us know.\r\n\r\nSupport of multi-gpu with Torch Distributed. Training at >x3 on 4GPU\r\nDue to requirement of Torch distributed, python3 is required for this new version\r\n\r\nChange from \"epoch\" to \"steps\": this has been discussed for a while and it is defitely better when working with big corpora.\r\n\r\nCrayon support has been removed.\r\n"},"comment":{"url":"https://api.github.com/repos/OpenNMT/OpenNMT-py/issues/comments/399359579","html_url":"https://github.com/OpenNMT/OpenNMT-py/pull/762#issuecomment-399359579","issue_url":"https://api.github.com/repos/OpenNMT/OpenNMT-py/issues/762","id":399359579,"node_id":"MDEyOklzc3VlQ29tbWVudDM5OTM1OTU3OQ==","user":{"login":"Gldkslfmsd","id":5314653,"node_id":"MDQ6VXNlcjUzMTQ2NTM=","avatar_url":"https://avatars2.githubusercontent.com/u/5314653?v=4","gravatar_id":"","url":"https://api.github.com/users/Gldkslfmsd","html_url":"https://github.com/Gldkslfmsd","followers_url":"https://api.github.com/users/Gldkslfmsd/followers","following_url":"https://api.github.com/users/Gldkslfmsd/following{/other_user}","gists_url":"https://api.github.com/users/Gldkslfmsd/gists{/gist_id}","starred_url":"https://api.github.com/users/Gldkslfmsd/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Gldkslfmsd/subscriptions","organizations_url":"https://api.github.com/users/Gldkslfmsd/orgs","repos_url":"https://api.github.com/users/Gldkslfmsd/repos","events_url":"https://api.github.com/users/Gldkslfmsd/events{/privacy}","received_events_url":"https://api.github.com/users/Gldkslfmsd/received_events","type":"User","site_admin":false},"created_at":"2018-06-22T08:09:16Z","updated_at":"2018-06-22T08:09:16Z","author_association":"NONE","body":"I have several comments/questions:\r\n- migrating epochs to steps is a good thing, but one wants to know both measures in log, or at least be able to reconstruct the epochs from steps easily. It's handy for analysis, for comparison between runs on same data with diferent settings etc. Will it be possible? \r\n- Also, what is the exact definition of \"training step\" here? How many training data are consumed in each step?\r\n- I would vote for -save_checkpoint_every_seconds in addition to -save_checkpoint_every_steps. \r\n- when you're doing so big refactoring, could you add a feature to log current time to prefix of each log message? It's good to see from one quick view the job is not stucked, or how long after the start it crashed, etc.\r\n\r\nThank you!"}}
{"actor":{"display_login":"Gldkslfmsd"}}