{"action":"reopened","number":4124,"pull_request":{"url":"https://api.github.com/repos/dmlc/mxnet/pulls/4124","id":96822215,"html_url":"https://github.com/dmlc/mxnet/pull/4124","diff_url":"https://github.com/dmlc/mxnet/pull/4124.diff","patch_url":"https://github.com/dmlc/mxnet/pull/4124.patch","issue_url":"https://api.github.com/repos/dmlc/mxnet/issues/4124","number":4124,"state":"open","locked":false,"title":"Checkpoint optimizer states","user":{"login":"zorksylar","id":1302872,"avatar_url":"https://avatars.githubusercontent.com/u/1302872?v=3","gravatar_id":"","url":"https://api.github.com/users/zorksylar","html_url":"https://github.com/zorksylar","followers_url":"https://api.github.com/users/zorksylar/followers","following_url":"https://api.github.com/users/zorksylar/following{/other_user}","gists_url":"https://api.github.com/users/zorksylar/gists{/gist_id}","starred_url":"https://api.github.com/users/zorksylar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zorksylar/subscriptions","organizations_url":"https://api.github.com/users/zorksylar/orgs","repos_url":"https://api.github.com/users/zorksylar/repos","events_url":"https://api.github.com/users/zorksylar/events{/privacy}","received_events_url":"https://api.github.com/users/zorksylar/received_events","type":"User","site_admin":false},"body":"Currently, mxnet only checkpoint the model. In practice, the computation states are not only model but also states of the optimizer. For SGD with momentum enabled, momentum is the state of the SGD optimizer, and it saves something like `history of gradient`. So, when checkpointing, the states of optimizer should be saved as well.\r\n\r\nIf the optimizer state is not saved, when load a checkpoint and continue training, there will be a sharp decline for both training-accuracy and validation-accuracy.\r\n\r\nThis PR saves optimizer states when checkpointing.","created_at":"2016-12-06T22:43:05Z","updated_at":"2016-12-28T20:37:12Z","closed_at":null,"merged_at":null,"merge_commit_sha":"d4f5cd77032b599f194530fbb2a2537e93f0ba13","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/dmlc/mxnet/pulls/4124/commits","review_comments_url":"https://api.github.com/repos/dmlc/mxnet/pulls/4124/comments","review_comment_url":"https://api.github.com/repos/dmlc/mxnet/pulls/comments{/number}","comments_url":"https://api.github.com/repos/dmlc/mxnet/issues/4124/comments","statuses_url":"https://api.github.com/repos/dmlc/mxnet/statuses/8df19a31743e9a606eac36bc0b1fbdd6580f8c63","head":{"label":"zorksylar:checkpoint_optimizer_states","ref":"checkpoint_optimizer_states","sha":"8df19a31743e9a606eac36bc0b1fbdd6580f8c63","user":{"login":"zorksylar","id":1302872,"avatar_url":"https://avatars.githubusercontent.com/u/1302872?v=3","gravatar_id":"","url":"https://api.github.com/users/zorksylar","html_url":"https://github.com/zorksylar","followers_url":"https://api.github.com/users/zorksylar/followers","following_url":"https://api.github.com/users/zorksylar/following{/other_user}","gists_url":"https://api.github.com/users/zorksylar/gists{/gist_id}","starred_url":"https://api.github.com/users/zorksylar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zorksylar/subscriptions","organizations_url":"https://api.github.com/users/zorksylar/orgs","repos_url":"https://api.github.com/users/zorksylar/repos","events_url":"https://api.github.com/users/zorksylar/events{/privacy}","received_events_url":"https://api.github.com/users/zorksylar/received_events","type":"User","site_admin":false},"repo":{"id":72238741,"name":"mxnet","full_name":"zorksylar/mxnet","owner":{"login":"zorksylar","id":1302872,"avatar_url":"https://avatars.githubusercontent.com/u/1302872?v=3","gravatar_id":"","url":"https://api.github.com/users/zorksylar","html_url":"https://github.com/zorksylar","followers_url":"https://api.github.com/users/zorksylar/followers","following_url":"https://api.github.com/users/zorksylar/following{/other_user}","gists_url":"https://api.github.com/users/zorksylar/gists{/gist_id}","starred_url":"https://api.github.com/users/zorksylar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zorksylar/subscriptions","organizations_url":"https://api.github.com/users/zorksylar/orgs","repos_url":"https://api.github.com/users/zorksylar/repos","events_url":"https://api.github.com/users/zorksylar/events{/privacy}","received_events_url":"https://api.github.com/users/zorksylar/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/zorksylar/mxnet","description":"Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more","fork":true,"url":"https://api.github.com/repos/zorksylar/mxnet","forks_url":"https://api.github.com/repos/zorksylar/mxnet/forks","keys_url":"https://api.github.com/repos/zorksylar/mxnet/keys{/key_id}","collaborators_url":"https://api.github.com/repos/zorksylar/mxnet/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/zorksylar/mxnet/teams","hooks_url":"https://api.github.com/repos/zorksylar/mxnet/hooks","issue_events_url":"https://api.github.com/repos/zorksylar/mxnet/issues/events{/number}","events_url":"https://api.github.com/repos/zorksylar/mxnet/events","assignees_url":"https://api.github.com/repos/zorksylar/mxnet/assignees{/user}","branches_url":"https://api.github.com/repos/zorksylar/mxnet/branches{/branch}","tags_url":"https://api.github.com/repos/zorksylar/mxnet/tags","blobs_url":"https://api.github.com/repos/zorksylar/mxnet/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/zorksylar/mxnet/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/zorksylar/mxnet/git/refs{/sha}","trees_url":"https://api.github.com/repos/zorksylar/mxnet/git/trees{/sha}","statuses_url":"https://api.github.com/repos/zorksylar/mxnet/statuses/{sha}","languages_url":"https://api.github.com/repos/zorksylar/mxnet/languages","stargazers_url":"https://api.github.com/repos/zorksylar/mxnet/stargazers","contributors_url":"https://api.github.com/repos/zorksylar/mxnet/contributors","subscribers_url":"https://api.github.com/repos/zorksylar/mxnet/subscribers","subscription_url":"https://api.github.com/repos/zorksylar/mxnet/subscription","commits_url":"https://api.github.com/repos/zorksylar/mxnet/commits{/sha}","git_commits_url":"https://api.github.com/repos/zorksylar/mxnet/git/commits{/sha}","comments_url":"https://api.github.com/repos/zorksylar/mxnet/comments{/number}","issue_comment_url":"https://api.github.com/repos/zorksylar/mxnet/issues/comments{/number}","contents_url":"https://api.github.com/repos/zorksylar/mxnet/contents/{+path}","compare_url":"https://api.github.com/repos/zorksylar/mxnet/compare/{base}...{head}","merges_url":"https://api.github.com/repos/zorksylar/mxnet/merges","archive_url":"https://api.github.com/repos/zorksylar/mxnet/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/zorksylar/mxnet/downloads","issues_url":"https://api.github.com/repos/zorksylar/mxnet/issues{/number}","pulls_url":"https://api.github.com/repos/zorksylar/mxnet/pulls{/number}","milestones_url":"https://api.github.com/repos/zorksylar/mxnet/milestones{/number}","notifications_url":"https://api.github.com/repos/zorksylar/mxnet/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/zorksylar/mxnet/labels{/name}","releases_url":"https://api.github.com/repos/zorksylar/mxnet/releases{/id}","deployments_url":"https://api.github.com/repos/zorksylar/mxnet/deployments","created_at":"2016-10-28T20:11:09Z","updated_at":"2016-12-02T01:57:25Z","pushed_at":"2016-12-28T01:37:21Z","git_url":"git://github.com/zorksylar/mxnet.git","ssh_url":"git@github.com:zorksylar/mxnet.git","clone_url":"https://github.com/zorksylar/mxnet.git","svn_url":"https://github.com/zorksylar/mxnet","homepage":"http://mxnet.io","size":12590,"stargazers_count":0,"watchers_count":0,"language":"Jupyter Notebook","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"dmlc:master","ref":"master","sha":"f92ebf12156dec8ab5a8deafa3514ef42633b3f7","user":{"login":"dmlc","id":11508361,"avatar_url":"https://avatars.githubusercontent.com/u/11508361?v=3","gravatar_id":"","url":"https://api.github.com/users/dmlc","html_url":"https://github.com/dmlc","followers_url":"https://api.github.com/users/dmlc/followers","following_url":"https://api.github.com/users/dmlc/following{/other_user}","gists_url":"https://api.github.com/users/dmlc/gists{/gist_id}","starred_url":"https://api.github.com/users/dmlc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dmlc/subscriptions","organizations_url":"https://api.github.com/users/dmlc/orgs","repos_url":"https://api.github.com/users/dmlc/repos","events_url":"https://api.github.com/users/dmlc/events{/privacy}","received_events_url":"https://api.github.com/users/dmlc/received_events","type":"Organization","site_admin":false},"repo":{"id":34864402,"name":"mxnet","full_name":"dmlc/mxnet","owner":{"login":"dmlc","id":11508361,"avatar_url":"https://avatars.githubusercontent.com/u/11508361?v=3","gravatar_id":"","url":"https://api.github.com/users/dmlc","html_url":"https://github.com/dmlc","followers_url":"https://api.github.com/users/dmlc/followers","following_url":"https://api.github.com/users/dmlc/following{/other_user}","gists_url":"https://api.github.com/users/dmlc/gists{/gist_id}","starred_url":"https://api.github.com/users/dmlc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dmlc/subscriptions","organizations_url":"https://api.github.com/users/dmlc/orgs","repos_url":"https://api.github.com/users/dmlc/repos","events_url":"https://api.github.com/users/dmlc/events{/privacy}","received_events_url":"https://api.github.com/users/dmlc/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/dmlc/mxnet","description":"Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more","fork":false,"url":"https://api.github.com/repos/dmlc/mxnet","forks_url":"https://api.github.com/repos/dmlc/mxnet/forks","keys_url":"https://api.github.com/repos/dmlc/mxnet/keys{/key_id}","collaborators_url":"https://api.github.com/repos/dmlc/mxnet/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/dmlc/mxnet/teams","hooks_url":"https://api.github.com/repos/dmlc/mxnet/hooks","issue_events_url":"https://api.github.com/repos/dmlc/mxnet/issues/events{/number}","events_url":"https://api.github.com/repos/dmlc/mxnet/events","assignees_url":"https://api.github.com/repos/dmlc/mxnet/assignees{/user}","branches_url":"https://api.github.com/repos/dmlc/mxnet/branches{/branch}","tags_url":"https://api.github.com/repos/dmlc/mxnet/tags","blobs_url":"https://api.github.com/repos/dmlc/mxnet/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/dmlc/mxnet/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/dmlc/mxnet/git/refs{/sha}","trees_url":"https://api.github.com/repos/dmlc/mxnet/git/trees{/sha}","statuses_url":"https://api.github.com/repos/dmlc/mxnet/statuses/{sha}","languages_url":"https://api.github.com/repos/dmlc/mxnet/languages","stargazers_url":"https://api.github.com/repos/dmlc/mxnet/stargazers","contributors_url":"https://api.github.com/repos/dmlc/mxnet/contributors","subscribers_url":"https://api.github.com/repos/dmlc/mxnet/subscribers","subscription_url":"https://api.github.com/repos/dmlc/mxnet/subscription","commits_url":"https://api.github.com/repos/dmlc/mxnet/commits{/sha}","git_commits_url":"https://api.github.com/repos/dmlc/mxnet/git/commits{/sha}","comments_url":"https://api.github.com/repos/dmlc/mxnet/comments{/number}","issue_comment_url":"https://api.github.com/repos/dmlc/mxnet/issues/comments{/number}","contents_url":"https://api.github.com/repos/dmlc/mxnet/contents/{+path}","compare_url":"https://api.github.com/repos/dmlc/mxnet/compare/{base}...{head}","merges_url":"https://api.github.com/repos/dmlc/mxnet/merges","archive_url":"https://api.github.com/repos/dmlc/mxnet/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/dmlc/mxnet/downloads","issues_url":"https://api.github.com/repos/dmlc/mxnet/issues{/number}","pulls_url":"https://api.github.com/repos/dmlc/mxnet/pulls{/number}","milestones_url":"https://api.github.com/repos/dmlc/mxnet/milestones{/number}","notifications_url":"https://api.github.com/repos/dmlc/mxnet/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/dmlc/mxnet/labels{/name}","releases_url":"https://api.github.com/repos/dmlc/mxnet/releases{/id}","deployments_url":"https://api.github.com/repos/dmlc/mxnet/deployments","created_at":"2015-04-30T16:21:15Z","updated_at":"2016-12-28T16:00:58Z","pushed_at":"2016-12-28T20:18:30Z","git_url":"git://github.com/dmlc/mxnet.git","ssh_url":"git@github.com:dmlc/mxnet.git","clone_url":"https://github.com/dmlc/mxnet.git","svn_url":"https://github.com/dmlc/mxnet","homepage":"http://mxnet.io","size":12843,"stargazers_count":7338,"watchers_count":7338,"language":"C++","has_issues":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":2720,"mirror_url":null,"open_issues_count":983,"forks":2720,"open_issues":983,"watchers":7338,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/dmlc/mxnet/pulls/4124"},"html":{"href":"https://github.com/dmlc/mxnet/pull/4124"},"issue":{"href":"https://api.github.com/repos/dmlc/mxnet/issues/4124"},"comments":{"href":"https://api.github.com/repos/dmlc/mxnet/issues/4124/comments"},"review_comments":{"href":"https://api.github.com/repos/dmlc/mxnet/pulls/4124/comments"},"review_comment":{"href":"https://api.github.com/repos/dmlc/mxnet/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/dmlc/mxnet/pulls/4124/commits"},"statuses":{"href":"https://api.github.com/repos/dmlc/mxnet/statuses/8df19a31743e9a606eac36bc0b1fbdd6580f8c63"}},"merged":false,"mergeable":null,"mergeable_state":"unknown","merged_by":null,"comments":2,"review_comments":0,"maintainer_can_modify":true,"commits":8,"additions":66,"deletions":19,"changed_files":4}}
{"actor":{"display_login":"zorksylar"}}