{"action":"created","comment":{"url":"https://api.github.com/repos/PaddlePaddle/Paddle/pulls/comments/241061872","pull_request_review_id":184237181,"id":241061872,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDI0MTA2MTg3Mg==","diff_hunk":"@@ -84,6 +91,31 @@ def append_regularization_ops(parameters_and_grads, regularization=None):\n     return params_and_grads\n \n \n+@contextlib.contextmanager\n+def append_weight_decay(param_and_grads, weight_decay=None):\n+    \"\"\"Add decay for the weight.\n+\n+    Appends weight decay operators in the BlockDesc.\n+    This will update the optimized parameters by using the\n+    parameters before optimization.","path":"python/paddle/fluid/regularizer.py","position":31,"original_position":31,"commit_id":"104e69ad4fd559fc86c8ae24ace340245928852e","original_commit_id":"b988991d468cd1fddebc526cc7b26ebe8430bb73","user":{"login":"chengduoZH","id":30176695,"node_id":"MDQ6VXNlcjMwMTc2Njk1","avatar_url":"https://avatars2.githubusercontent.com/u/30176695?v=4","gravatar_id":"","url":"https://api.github.com/users/chengduoZH","html_url":"https://github.com/chengduoZH","followers_url":"https://api.github.com/users/chengduoZH/followers","following_url":"https://api.github.com/users/chengduoZH/following{/other_user}","gists_url":"https://api.github.com/users/chengduoZH/gists{/gist_id}","starred_url":"https://api.github.com/users/chengduoZH/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chengduoZH/subscriptions","organizations_url":"https://api.github.com/users/chengduoZH/orgs","repos_url":"https://api.github.com/users/chengduoZH/repos","events_url":"https://api.github.com/users/chengduoZH/events{/privacy}","received_events_url":"https://api.github.com/users/chengduoZH/received_events","type":"User","site_admin":false},"body":"Done","created_at":"2018-12-12T15:32:28Z","updated_at":"2018-12-12T15:32:41Z","html_url":"https://github.com/PaddlePaddle/Paddle/pull/14828#discussion_r241061872","pull_request_url":"https://api.github.com/repos/PaddlePaddle/Paddle/pulls/14828","author_association":"CONTRIBUTOR","_links":{"self":{"href":"https://api.github.com/repos/PaddlePaddle/Paddle/pulls/comments/241061872"},"html":{"href":"https://github.com/PaddlePaddle/Paddle/pull/14828#discussion_r241061872"},"pull_request":{"href":"https://api.github.com/repos/PaddlePaddle/Paddle/pulls/14828"}},"in_reply_to_id":240992673},"pull_request":{"url":"https://api.github.com/repos/PaddlePaddle/Paddle/pulls/14828","id":237249474,"node_id":"MDExOlB1bGxSZXF1ZXN0MjM3MjQ5NDc0","html_url":"https://github.com/PaddlePaddle/Paddle/pull/14828","diff_url":"https://github.com/PaddlePaddle/Paddle/pull/14828.diff","patch_url":"https://github.com/PaddlePaddle/Paddle/pull/14828.patch","issue_url":"https://api.github.com/repos/PaddlePaddle/Paddle/issues/14828","number":14828,"state":"open","locked":false,"title":"Add weight decay","user":{"login":"chengduoZH","id":30176695,"node_id":"MDQ6VXNlcjMwMTc2Njk1","avatar_url":"https://avatars2.githubusercontent.com/u/30176695?v=4","gravatar_id":"","url":"https://api.github.com/users/chengduoZH","html_url":"https://github.com/chengduoZH","followers_url":"https://api.github.com/users/chengduoZH/followers","following_url":"https://api.github.com/users/chengduoZH/following{/other_user}","gists_url":"https://api.github.com/users/chengduoZH/gists{/gist_id}","starred_url":"https://api.github.com/users/chengduoZH/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chengduoZH/subscriptions","organizations_url":"https://api.github.com/users/chengduoZH/orgs","repos_url":"https://api.github.com/users/chengduoZH/repos","events_url":"https://api.github.com/users/chengduoZH/events{/privacy}","received_events_url":"https://api.github.com/users/chengduoZH/received_events","type":"User","site_admin":false},"body":"fix https://github.com/PaddlePaddle/Paddle/issues/5901\r\nfix https://github.com/PaddlePaddle/Paddle/issues/14812\r\n\r\nJust adding the square of the weights to the loss function is **not** the correct way of using L2 regularization/weight decay with Adam, since that will interact with the **Moment1** and **Moment2** parameters in strange ways.","created_at":"2018-12-10T09:27:19Z","updated_at":"2018-12-12T15:32:41Z","closed_at":null,"merged_at":null,"merge_commit_sha":"5698633fa01b7c9de2e948e6b222b4c54816d871","assignee":null,"assignees":[],"requested_reviewers":[{"login":"kuke","id":3064195,"node_id":"MDQ6VXNlcjMwNjQxOTU=","avatar_url":"https://avatars2.githubusercontent.com/u/3064195?v=4","gravatar_id":"","url":"https://api.github.com/users/kuke","html_url":"https://github.com/kuke","followers_url":"https://api.github.com/users/kuke/followers","following_url":"https://api.github.com/users/kuke/following{/other_user}","gists_url":"https://api.github.com/users/kuke/gists{/gist_id}","starred_url":"https://api.github.com/users/kuke/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kuke/subscriptions","organizations_url":"https://api.github.com/users/kuke/orgs","repos_url":"https://api.github.com/users/kuke/repos","events_url":"https://api.github.com/users/kuke/events{/privacy}","received_events_url":"https://api.github.com/users/kuke/received_events","type":"User","site_admin":false},{"login":"qingqing01","id":7845005,"node_id":"MDQ6VXNlcjc4NDUwMDU=","avatar_url":"https://avatars0.githubusercontent.com/u/7845005?v=4","gravatar_id":"","url":"https://api.github.com/users/qingqing01","html_url":"https://github.com/qingqing01","followers_url":"https://api.github.com/users/qingqing01/followers","following_url":"https://api.github.com/users/qingqing01/following{/other_user}","gists_url":"https://api.github.com/users/qingqing01/gists{/gist_id}","starred_url":"https://api.github.com/users/qingqing01/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/qingqing01/subscriptions","organizations_url":"https://api.github.com/users/qingqing01/orgs","repos_url":"https://api.github.com/users/qingqing01/repos","events_url":"https://api.github.com/users/qingqing01/events{/privacy}","received_events_url":"https://api.github.com/users/qingqing01/received_events","type":"User","site_admin":false}],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/PaddlePaddle/Paddle/pulls/14828/commits","review_comments_url":"https://api.github.com/repos/PaddlePaddle/Paddle/pulls/14828/comments","review_comment_url":"https://api.github.com/repos/PaddlePaddle/Paddle/pulls/comments{/number}","comments_url":"https://api.github.com/repos/PaddlePaddle/Paddle/issues/14828/comments","statuses_url":"https://api.github.com/repos/PaddlePaddle/Paddle/statuses/104e69ad4fd559fc86c8ae24ace340245928852e","head":{"label":"chengduoZH:add_weight_decay","ref":"add_weight_decay","sha":"104e69ad4fd559fc86c8ae24ace340245928852e","user":{"login":"chengduoZH","id":30176695,"node_id":"MDQ6VXNlcjMwMTc2Njk1","avatar_url":"https://avatars2.githubusercontent.com/u/30176695?v=4","gravatar_id":"","url":"https://api.github.com/users/chengduoZH","html_url":"https://github.com/chengduoZH","followers_url":"https://api.github.com/users/chengduoZH/followers","following_url":"https://api.github.com/users/chengduoZH/following{/other_user}","gists_url":"https://api.github.com/users/chengduoZH/gists{/gist_id}","starred_url":"https://api.github.com/users/chengduoZH/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chengduoZH/subscriptions","organizations_url":"https://api.github.com/users/chengduoZH/orgs","repos_url":"https://api.github.com/users/chengduoZH/repos","events_url":"https://api.github.com/users/chengduoZH/events{/privacy}","received_events_url":"https://api.github.com/users/chengduoZH/received_events","type":"User","site_admin":false},"repo":{"id":99756501,"node_id":"MDEwOlJlcG9zaXRvcnk5OTc1NjUwMQ==","name":"Paddle","full_name":"chengduoZH/Paddle","private":false,"owner":{"login":"chengduoZH","id":30176695,"node_id":"MDQ6VXNlcjMwMTc2Njk1","avatar_url":"https://avatars2.githubusercontent.com/u/30176695?v=4","gravatar_id":"","url":"https://api.github.com/users/chengduoZH","html_url":"https://github.com/chengduoZH","followers_url":"https://api.github.com/users/chengduoZH/followers","following_url":"https://api.github.com/users/chengduoZH/following{/other_user}","gists_url":"https://api.github.com/users/chengduoZH/gists{/gist_id}","starred_url":"https://api.github.com/users/chengduoZH/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chengduoZH/subscriptions","organizations_url":"https://api.github.com/users/chengduoZH/orgs","repos_url":"https://api.github.com/users/chengduoZH/repos","events_url":"https://api.github.com/users/chengduoZH/events{/privacy}","received_events_url":"https://api.github.com/users/chengduoZH/received_events","type":"User","site_admin":false},"html_url":"https://github.com/chengduoZH/Paddle","description":"PArallel Distributed Deep LEarning","fork":true,"url":"https://api.github.com/repos/chengduoZH/Paddle","forks_url":"https://api.github.com/repos/chengduoZH/Paddle/forks","keys_url":"https://api.github.com/repos/chengduoZH/Paddle/keys{/key_id}","collaborators_url":"https://api.github.com/repos/chengduoZH/Paddle/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/chengduoZH/Paddle/teams","hooks_url":"https://api.github.com/repos/chengduoZH/Paddle/hooks","issue_events_url":"https://api.github.com/repos/chengduoZH/Paddle/issues/events{/number}","events_url":"https://api.github.com/repos/chengduoZH/Paddle/events","assignees_url":"https://api.github.com/repos/chengduoZH/Paddle/assignees{/user}","branches_url":"https://api.github.com/repos/chengduoZH/Paddle/branches{/branch}","tags_url":"https://api.github.com/repos/chengduoZH/Paddle/tags","blobs_url":"https://api.github.com/repos/chengduoZH/Paddle/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/chengduoZH/Paddle/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/chengduoZH/Paddle/git/refs{/sha}","trees_url":"https://api.github.com/repos/chengduoZH/Paddle/git/trees{/sha}","statuses_url":"https://api.github.com/repos/chengduoZH/Paddle/statuses/{sha}","languages_url":"https://api.github.com/repos/chengduoZH/Paddle/languages","stargazers_url":"https://api.github.com/repos/chengduoZH/Paddle/stargazers","contributors_url":"https://api.github.com/repos/chengduoZH/Paddle/contributors","subscribers_url":"https://api.github.com/repos/chengduoZH/Paddle/subscribers","subscription_url":"https://api.github.com/repos/chengduoZH/Paddle/subscription","commits_url":"https://api.github.com/repos/chengduoZH/Paddle/commits{/sha}","git_commits_url":"https://api.github.com/repos/chengduoZH/Paddle/git/commits{/sha}","comments_url":"https://api.github.com/repos/chengduoZH/Paddle/comments{/number}","issue_comment_url":"https://api.github.com/repos/chengduoZH/Paddle/issues/comments{/number}","contents_url":"https://api.github.com/repos/chengduoZH/Paddle/contents/{+path}","compare_url":"https://api.github.com/repos/chengduoZH/Paddle/compare/{base}...{head}","merges_url":"https://api.github.com/repos/chengduoZH/Paddle/merges","archive_url":"https://api.github.com/repos/chengduoZH/Paddle/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/chengduoZH/Paddle/downloads","issues_url":"https://api.github.com/repos/chengduoZH/Paddle/issues{/number}","pulls_url":"https://api.github.com/repos/chengduoZH/Paddle/pulls{/number}","milestones_url":"https://api.github.com/repos/chengduoZH/Paddle/milestones{/number}","notifications_url":"https://api.github.com/repos/chengduoZH/Paddle/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/chengduoZH/Paddle/labels{/name}","releases_url":"https://api.github.com/repos/chengduoZH/Paddle/releases{/id}","deployments_url":"https://api.github.com/repos/chengduoZH/Paddle/deployments","created_at":"2017-08-09T02:26:38Z","updated_at":"2018-12-11T03:38:44Z","pushed_at":"2018-12-12T15:31:56Z","git_url":"git://github.com/chengduoZH/Paddle.git","ssh_url":"git@github.com:chengduoZH/Paddle.git","clone_url":"https://github.com/chengduoZH/Paddle.git","svn_url":"https://github.com/chengduoZH/Paddle","homepage":"http://www.paddlepaddle.org/","size":116105,"stargazers_count":0,"watchers_count":0,"language":"C++","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":1,"mirror_url":null,"archived":false,"open_issues_count":0,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":1,"open_issues":0,"watchers":0,"default_branch":"develop"}},"base":{"label":"PaddlePaddle:develop","ref":"develop","sha":"30aad88449f1f671cbc80f006c13eced9b2bca33","user":{"login":"PaddlePaddle","id":23534030,"node_id":"MDEyOk9yZ2FuaXphdGlvbjIzNTM0MDMw","avatar_url":"https://avatars3.githubusercontent.com/u/23534030?v=4","gravatar_id":"","url":"https://api.github.com/users/PaddlePaddle","html_url":"https://github.com/PaddlePaddle","followers_url":"https://api.github.com/users/PaddlePaddle/followers","following_url":"https://api.github.com/users/PaddlePaddle/following{/other_user}","gists_url":"https://api.github.com/users/PaddlePaddle/gists{/gist_id}","starred_url":"https://api.github.com/users/PaddlePaddle/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/PaddlePaddle/subscriptions","organizations_url":"https://api.github.com/users/PaddlePaddle/orgs","repos_url":"https://api.github.com/users/PaddlePaddle/repos","events_url":"https://api.github.com/users/PaddlePaddle/events{/privacy}","received_events_url":"https://api.github.com/users/PaddlePaddle/received_events","type":"Organization","site_admin":false},"repo":{"id":65711522,"node_id":"MDEwOlJlcG9zaXRvcnk2NTcxMTUyMg==","name":"Paddle","full_name":"PaddlePaddle/Paddle","private":false,"owner":{"login":"PaddlePaddle","id":23534030,"node_id":"MDEyOk9yZ2FuaXphdGlvbjIzNTM0MDMw","avatar_url":"https://avatars3.githubusercontent.com/u/23534030?v=4","gravatar_id":"","url":"https://api.github.com/users/PaddlePaddle","html_url":"https://github.com/PaddlePaddle","followers_url":"https://api.github.com/users/PaddlePaddle/followers","following_url":"https://api.github.com/users/PaddlePaddle/following{/other_user}","gists_url":"https://api.github.com/users/PaddlePaddle/gists{/gist_id}","starred_url":"https://api.github.com/users/PaddlePaddle/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/PaddlePaddle/subscriptions","organizations_url":"https://api.github.com/users/PaddlePaddle/orgs","repos_url":"https://api.github.com/users/PaddlePaddle/repos","events_url":"https://api.github.com/users/PaddlePaddle/events{/privacy}","received_events_url":"https://api.github.com/users/PaddlePaddle/received_events","type":"Organization","site_admin":false},"html_url":"https://github.com/PaddlePaddle/Paddle","description":"PArallel Distributed Deep LEarning （PaddlePaddle核心框架，高性能单机、分布式训练和跨平台部署）","fork":false,"url":"https://api.github.com/repos/PaddlePaddle/Paddle","forks_url":"https://api.github.com/repos/PaddlePaddle/Paddle/forks","keys_url":"https://api.github.com/repos/PaddlePaddle/Paddle/keys{/key_id}","collaborators_url":"https://api.github.com/repos/PaddlePaddle/Paddle/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/PaddlePaddle/Paddle/teams","hooks_url":"https://api.github.com/repos/PaddlePaddle/Paddle/hooks","issue_events_url":"https://api.github.com/repos/PaddlePaddle/Paddle/issues/events{/number}","events_url":"https://api.github.com/repos/PaddlePaddle/Paddle/events","assignees_url":"https://api.github.com/repos/PaddlePaddle/Paddle/assignees{/user}","branches_url":"https://api.github.com/repos/PaddlePaddle/Paddle/branches{/branch}","tags_url":"https://api.github.com/repos/PaddlePaddle/Paddle/tags","blobs_url":"https://api.github.com/repos/PaddlePaddle/Paddle/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/PaddlePaddle/Paddle/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/PaddlePaddle/Paddle/git/refs{/sha}","trees_url":"https://api.github.com/repos/PaddlePaddle/Paddle/git/trees{/sha}","statuses_url":"https://api.github.com/repos/PaddlePaddle/Paddle/statuses/{sha}","languages_url":"https://api.github.com/repos/PaddlePaddle/Paddle/languages","stargazers_url":"https://api.github.com/repos/PaddlePaddle/Paddle/stargazers","contributors_url":"https://api.github.com/repos/PaddlePaddle/Paddle/contributors","subscribers_url":"https://api.github.com/repos/PaddlePaddle/Paddle/subscribers","subscription_url":"https://api.github.com/repos/PaddlePaddle/Paddle/subscription","commits_url":"https://api.github.com/repos/PaddlePaddle/Paddle/commits{/sha}","git_commits_url":"https://api.github.com/repos/PaddlePaddle/Paddle/git/commits{/sha}","comments_url":"https://api.github.com/repos/PaddlePaddle/Paddle/comments{/number}","issue_comment_url":"https://api.github.com/repos/PaddlePaddle/Paddle/issues/comments{/number}","contents_url":"https://api.github.com/repos/PaddlePaddle/Paddle/contents/{+path}","compare_url":"https://api.github.com/repos/PaddlePaddle/Paddle/compare/{base}...{head}","merges_url":"https://api.github.com/repos/PaddlePaddle/Paddle/merges","archive_url":"https://api.github.com/repos/PaddlePaddle/Paddle/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/PaddlePaddle/Paddle/downloads","issues_url":"https://api.github.com/repos/PaddlePaddle/Paddle/issues{/number}","pulls_url":"https://api.github.com/repos/PaddlePaddle/Paddle/pulls{/number}","milestones_url":"https://api.github.com/repos/PaddlePaddle/Paddle/milestones{/number}","notifications_url":"https://api.github.com/repos/PaddlePaddle/Paddle/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/PaddlePaddle/Paddle/labels{/name}","releases_url":"https://api.github.com/repos/PaddlePaddle/Paddle/releases{/id}","deployments_url":"https://api.github.com/repos/PaddlePaddle/Paddle/deployments","created_at":"2016-08-15T06:59:08Z","updated_at":"2018-12-12T13:51:48Z","pushed_at":"2018-12-12T15:31:59Z","git_url":"git://github.com/PaddlePaddle/Paddle.git","ssh_url":"git@github.com:PaddlePaddle/Paddle.git","clone_url":"https://github.com/PaddlePaddle/Paddle.git","svn_url":"https://github.com/PaddlePaddle/Paddle","homepage":"http://www.paddlepaddle.org/","size":142905,"stargazers_count":7888,"watchers_count":7888,"language":"C++","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":2145,"mirror_url":null,"archived":false,"open_issues_count":744,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":2145,"open_issues":744,"watchers":7888,"default_branch":"develop"}},"_links":{"self":{"href":"https://api.github.com/repos/PaddlePaddle/Paddle/pulls/14828"},"html":{"href":"https://github.com/PaddlePaddle/Paddle/pull/14828"},"issue":{"href":"https://api.github.com/repos/PaddlePaddle/Paddle/issues/14828"},"comments":{"href":"https://api.github.com/repos/PaddlePaddle/Paddle/issues/14828/comments"},"review_comments":{"href":"https://api.github.com/repos/PaddlePaddle/Paddle/pulls/14828/comments"},"review_comment":{"href":"https://api.github.com/repos/PaddlePaddle/Paddle/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/PaddlePaddle/Paddle/pulls/14828/commits"},"statuses":{"href":"https://api.github.com/repos/PaddlePaddle/Paddle/statuses/104e69ad4fd559fc86c8ae24ace340245928852e"}},"author_association":"CONTRIBUTOR"}}
{"actor":{"display_login":"chengduoZH"}}