{"action":"closed","number":63,"pull_request":{"url":"https://api.github.com/repos/dpressel/baseline/pulls/63","id":192542484,"node_id":"MDExOlB1bGxSZXF1ZXN0MTkyNTQyNDg0","html_url":"https://github.com/dpressel/baseline/pull/63","diff_url":"https://github.com/dpressel/baseline/pull/63.diff","patch_url":"https://github.com/dpressel/baseline/pull/63.patch","issue_url":"https://api.github.com/repos/dpressel/baseline/issues/63","number":63,"state":"closed","locked":false,"title":"Added EMA training, loading, and exporting to tensorflow and tests","user":{"login":"blester125","id":10950530,"node_id":"MDQ6VXNlcjEwOTUwNTMw","avatar_url":"https://avatars3.githubusercontent.com/u/10950530?v=4","gravatar_id":"","url":"https://api.github.com/users/blester125","html_url":"https://github.com/blester125","followers_url":"https://api.github.com/users/blester125/followers","following_url":"https://api.github.com/users/blester125/following{/other_user}","gists_url":"https://api.github.com/users/blester125/gists{/gist_id}","starred_url":"https://api.github.com/users/blester125/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/blester125/subscriptions","organizations_url":"https://api.github.com/users/blester125/orgs","repos_url":"https://api.github.com/users/blester125/repos","events_url":"https://api.github.com/users/blester125/events{/privacy}","received_events_url":"https://api.github.com/users/blester125/received_events","type":"User","site_admin":false},"body":"The PR add Exponential Moving Average training to Tensorflow Classify.\r\n\r\nThe actual ema operations are added with the function `_add_ema` in tfy, ema calculation is done with a `tf.control_dependencies` in the trainer, and using the ema values is done with a special `tf.train.Saver` that reads the EMA values into the variable tensors.\r\n\r\nWhen saving the model a new `.eval_saver` is created. This can be used when restoring the model.\r\n\r\nThis is how the EMA works for training:\r\n * When training the model it creates a checkpoint at the end of each epoch. This has the current params and the ema version. \r\n * Before testing this checkpoint is restored using the special saver.\r\n * Evaluation is performed and the model is checkpointed for real if the performance is better.\r\n * The training checkpoint it then loaded and training continues.\r\n\r\nThis format allows for reloading the whole model for retraining, reload for patience, and things like that.\r\n\r\nThis PR also modifies the exporter so that if there is an `.eval_saver` present it loads in the EMA values.\r\n\r\nThe exporting was tested by exporting both EMA and normal models to make sure they work as well an an automated test that compares the saved values from a model trained with EMA.\r\n\r\nThe EMA is tested with many manual runs as well as automated testings that covers things like the emas are in the graph, get updated, can be saved and restored and the like.\r\n\r\nThere was also a small change to the exporter that allows for it to export a model created with random word vectors. This was tested by exporting untrained models with random vectors and pretrained and checking that the values in the embedding table were correct\r\n","created_at":"2018-06-04T22:13:01Z","updated_at":"2018-08-21T14:01:26Z","closed_at":"2018-08-21T14:01:26Z","merged_at":null,"merge_commit_sha":"bd1f48d8b749ccf9dbbba1ee4573ea6c57a50f87","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/dpressel/baseline/pulls/63/commits","review_comments_url":"https://api.github.com/repos/dpressel/baseline/pulls/63/comments","review_comment_url":"https://api.github.com/repos/dpressel/baseline/pulls/comments{/number}","comments_url":"https://api.github.com/repos/dpressel/baseline/issues/63/comments","statuses_url":"https://api.github.com/repos/dpressel/baseline/statuses/fe10a1aacaa93105969b25df7a533e8bcecb1c37","head":{"label":"blester125:ema-tf","ref":"ema-tf","sha":"fe10a1aacaa93105969b25df7a533e8bcecb1c37","user":{"login":"blester125","id":10950530,"node_id":"MDQ6VXNlcjEwOTUwNTMw","avatar_url":"https://avatars3.githubusercontent.com/u/10950530?v=4","gravatar_id":"","url":"https://api.github.com/users/blester125","html_url":"https://github.com/blester125","followers_url":"https://api.github.com/users/blester125/followers","following_url":"https://api.github.com/users/blester125/following{/other_user}","gists_url":"https://api.github.com/users/blester125/gists{/gist_id}","starred_url":"https://api.github.com/users/blester125/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/blester125/subscriptions","organizations_url":"https://api.github.com/users/blester125/orgs","repos_url":"https://api.github.com/users/blester125/repos","events_url":"https://api.github.com/users/blester125/events{/privacy}","received_events_url":"https://api.github.com/users/blester125/received_events","type":"User","site_admin":false},"repo":{"id":128435887,"node_id":"MDEwOlJlcG9zaXRvcnkxMjg0MzU4ODc=","name":"baseline","full_name":"blester125/baseline","owner":{"login":"blester125","id":10950530,"node_id":"MDQ6VXNlcjEwOTUwNTMw","avatar_url":"https://avatars3.githubusercontent.com/u/10950530?v=4","gravatar_id":"","url":"https://api.github.com/users/blester125","html_url":"https://github.com/blester125","followers_url":"https://api.github.com/users/blester125/followers","following_url":"https://api.github.com/users/blester125/following{/other_user}","gists_url":"https://api.github.com/users/blester125/gists{/gist_id}","starred_url":"https://api.github.com/users/blester125/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/blester125/subscriptions","organizations_url":"https://api.github.com/users/blester125/orgs","repos_url":"https://api.github.com/users/blester125/repos","events_url":"https://api.github.com/users/blester125/events{/privacy}","received_events_url":"https://api.github.com/users/blester125/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/blester125/baseline","description":"Strong Deep-Learning Baseline algorithms for NLP","fork":true,"url":"https://api.github.com/repos/blester125/baseline","forks_url":"https://api.github.com/repos/blester125/baseline/forks","keys_url":"https://api.github.com/repos/blester125/baseline/keys{/key_id}","collaborators_url":"https://api.github.com/repos/blester125/baseline/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/blester125/baseline/teams","hooks_url":"https://api.github.com/repos/blester125/baseline/hooks","issue_events_url":"https://api.github.com/repos/blester125/baseline/issues/events{/number}","events_url":"https://api.github.com/repos/blester125/baseline/events","assignees_url":"https://api.github.com/repos/blester125/baseline/assignees{/user}","branches_url":"https://api.github.com/repos/blester125/baseline/branches{/branch}","tags_url":"https://api.github.com/repos/blester125/baseline/tags","blobs_url":"https://api.github.com/repos/blester125/baseline/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/blester125/baseline/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/blester125/baseline/git/refs{/sha}","trees_url":"https://api.github.com/repos/blester125/baseline/git/trees{/sha}","statuses_url":"https://api.github.com/repos/blester125/baseline/statuses/{sha}","languages_url":"https://api.github.com/repos/blester125/baseline/languages","stargazers_url":"https://api.github.com/repos/blester125/baseline/stargazers","contributors_url":"https://api.github.com/repos/blester125/baseline/contributors","subscribers_url":"https://api.github.com/repos/blester125/baseline/subscribers","subscription_url":"https://api.github.com/repos/blester125/baseline/subscription","commits_url":"https://api.github.com/repos/blester125/baseline/commits{/sha}","git_commits_url":"https://api.github.com/repos/blester125/baseline/git/commits{/sha}","comments_url":"https://api.github.com/repos/blester125/baseline/comments{/number}","issue_comment_url":"https://api.github.com/repos/blester125/baseline/issues/comments{/number}","contents_url":"https://api.github.com/repos/blester125/baseline/contents/{+path}","compare_url":"https://api.github.com/repos/blester125/baseline/compare/{base}...{head}","merges_url":"https://api.github.com/repos/blester125/baseline/merges","archive_url":"https://api.github.com/repos/blester125/baseline/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/blester125/baseline/downloads","issues_url":"https://api.github.com/repos/blester125/baseline/issues{/number}","pulls_url":"https://api.github.com/repos/blester125/baseline/pulls{/number}","milestones_url":"https://api.github.com/repos/blester125/baseline/milestones{/number}","notifications_url":"https://api.github.com/repos/blester125/baseline/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/blester125/baseline/labels{/name}","releases_url":"https://api.github.com/repos/blester125/baseline/releases{/id}","deployments_url":"https://api.github.com/repos/blester125/baseline/deployments","created_at":"2018-04-06T18:28:46Z","updated_at":"2018-08-20T18:55:41Z","pushed_at":"2018-08-20T19:30:49Z","git_url":"git://github.com/blester125/baseline.git","ssh_url":"git@github.com:blester125/baseline.git","clone_url":"https://github.com/blester125/baseline.git","svn_url":"https://github.com/blester125/baseline","homepage":"","size":7060,"stargazers_count":0,"watchers_count":0,"language":"Python","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":0,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"dpressel:master","ref":"master","sha":"1136b7bc014d5f9df2cbddec1c1d2242e2395601","user":{"login":"dpressel","id":247881,"node_id":"MDQ6VXNlcjI0Nzg4MQ==","avatar_url":"https://avatars1.githubusercontent.com/u/247881?v=4","gravatar_id":"","url":"https://api.github.com/users/dpressel","html_url":"https://github.com/dpressel","followers_url":"https://api.github.com/users/dpressel/followers","following_url":"https://api.github.com/users/dpressel/following{/other_user}","gists_url":"https://api.github.com/users/dpressel/gists{/gist_id}","starred_url":"https://api.github.com/users/dpressel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dpressel/subscriptions","organizations_url":"https://api.github.com/users/dpressel/orgs","repos_url":"https://api.github.com/users/dpressel/repos","events_url":"https://api.github.com/users/dpressel/events{/privacy}","received_events_url":"https://api.github.com/users/dpressel/received_events","type":"User","site_admin":false},"repo":{"id":56975673,"node_id":"MDEwOlJlcG9zaXRvcnk1Njk3NTY3Mw==","name":"baseline","full_name":"dpressel/baseline","owner":{"login":"dpressel","id":247881,"node_id":"MDQ6VXNlcjI0Nzg4MQ==","avatar_url":"https://avatars1.githubusercontent.com/u/247881?v=4","gravatar_id":"","url":"https://api.github.com/users/dpressel","html_url":"https://github.com/dpressel","followers_url":"https://api.github.com/users/dpressel/followers","following_url":"https://api.github.com/users/dpressel/following{/other_user}","gists_url":"https://api.github.com/users/dpressel/gists{/gist_id}","starred_url":"https://api.github.com/users/dpressel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dpressel/subscriptions","organizations_url":"https://api.github.com/users/dpressel/orgs","repos_url":"https://api.github.com/users/dpressel/repos","events_url":"https://api.github.com/users/dpressel/events{/privacy}","received_events_url":"https://api.github.com/users/dpressel/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/dpressel/baseline","description":"Strong Deep-Learning Baseline algorithms for NLP","fork":false,"url":"https://api.github.com/repos/dpressel/baseline","forks_url":"https://api.github.com/repos/dpressel/baseline/forks","keys_url":"https://api.github.com/repos/dpressel/baseline/keys{/key_id}","collaborators_url":"https://api.github.com/repos/dpressel/baseline/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/dpressel/baseline/teams","hooks_url":"https://api.github.com/repos/dpressel/baseline/hooks","issue_events_url":"https://api.github.com/repos/dpressel/baseline/issues/events{/number}","events_url":"https://api.github.com/repos/dpressel/baseline/events","assignees_url":"https://api.github.com/repos/dpressel/baseline/assignees{/user}","branches_url":"https://api.github.com/repos/dpressel/baseline/branches{/branch}","tags_url":"https://api.github.com/repos/dpressel/baseline/tags","blobs_url":"https://api.github.com/repos/dpressel/baseline/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/dpressel/baseline/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/dpressel/baseline/git/refs{/sha}","trees_url":"https://api.github.com/repos/dpressel/baseline/git/trees{/sha}","statuses_url":"https://api.github.com/repos/dpressel/baseline/statuses/{sha}","languages_url":"https://api.github.com/repos/dpressel/baseline/languages","stargazers_url":"https://api.github.com/repos/dpressel/baseline/stargazers","contributors_url":"https://api.github.com/repos/dpressel/baseline/contributors","subscribers_url":"https://api.github.com/repos/dpressel/baseline/subscribers","subscription_url":"https://api.github.com/repos/dpressel/baseline/subscription","commits_url":"https://api.github.com/repos/dpressel/baseline/commits{/sha}","git_commits_url":"https://api.github.com/repos/dpressel/baseline/git/commits{/sha}","comments_url":"https://api.github.com/repos/dpressel/baseline/comments{/number}","issue_comment_url":"https://api.github.com/repos/dpressel/baseline/issues/comments{/number}","contents_url":"https://api.github.com/repos/dpressel/baseline/contents/{+path}","compare_url":"https://api.github.com/repos/dpressel/baseline/compare/{base}...{head}","merges_url":"https://api.github.com/repos/dpressel/baseline/merges","archive_url":"https://api.github.com/repos/dpressel/baseline/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/dpressel/baseline/downloads","issues_url":"https://api.github.com/repos/dpressel/baseline/issues{/number}","pulls_url":"https://api.github.com/repos/dpressel/baseline/pulls{/number}","milestones_url":"https://api.github.com/repos/dpressel/baseline/milestones{/number}","notifications_url":"https://api.github.com/repos/dpressel/baseline/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/dpressel/baseline/labels{/name}","releases_url":"https://api.github.com/repos/dpressel/baseline/releases{/id}","deployments_url":"https://api.github.com/repos/dpressel/baseline/deployments","created_at":"2016-04-24T14:23:02Z","updated_at":"2018-08-21T14:00:06Z","pushed_at":"2018-08-21T14:00:03Z","git_url":"git://github.com/dpressel/baseline.git","ssh_url":"git@github.com:dpressel/baseline.git","clone_url":"https://github.com/dpressel/baseline.git","svn_url":"https://github.com/dpressel/baseline","homepage":"","size":6335,"stargazers_count":151,"watchers_count":151,"language":"Python","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":39,"mirror_url":null,"archived":false,"open_issues_count":3,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":39,"open_issues":3,"watchers":151,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/dpressel/baseline/pulls/63"},"html":{"href":"https://github.com/dpressel/baseline/pull/63"},"issue":{"href":"https://api.github.com/repos/dpressel/baseline/issues/63"},"comments":{"href":"https://api.github.com/repos/dpressel/baseline/issues/63/comments"},"review_comments":{"href":"https://api.github.com/repos/dpressel/baseline/pulls/63/comments"},"review_comment":{"href":"https://api.github.com/repos/dpressel/baseline/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/dpressel/baseline/pulls/63/commits"},"statuses":{"href":"https://api.github.com/repos/dpressel/baseline/statuses/fe10a1aacaa93105969b25df7a533e8bcecb1c37"}},"author_association":"CONTRIBUTOR","merged":false,"mergeable":false,"rebaseable":false,"mergeable_state":"dirty","merged_by":null,"comments":1,"review_comments":0,"maintainer_can_modify":false,"commits":1,"additions":406,"deletions":43,"changed_files":18}}
{"actor":{"display_login":"blester125"}}