{"action":"closed","number":1554,"pull_request":{"url":"https://api.github.com/repos/dmlc/mxnet/pulls/1554","id":61341747,"html_url":"https://github.com/dmlc/mxnet/pull/1554","diff_url":"https://github.com/dmlc/mxnet/pull/1554.diff","patch_url":"https://github.com/dmlc/mxnet/pull/1554.patch","issue_url":"https://api.github.com/repos/dmlc/mxnet/issues/1554","number":1554,"state":"closed","locked":false,"title":"Fix #1323. lr_scheduler, clip_gradient, Xavier initializer are supported for R package","user":{"login":"Puriney","id":1233372,"avatar_url":"https://avatars.githubusercontent.com/u/1233372?v=3","gravatar_id":"","url":"https://api.github.com/users/Puriney","html_url":"https://github.com/Puriney","followers_url":"https://api.github.com/users/Puriney/followers","following_url":"https://api.github.com/users/Puriney/following{/other_user}","gists_url":"https://api.github.com/users/Puriney/gists{/gist_id}","starred_url":"https://api.github.com/users/Puriney/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Puriney/subscriptions","organizations_url":"https://api.github.com/users/Puriney/orgs","repos_url":"https://api.github.com/users/Puriney/repos","events_url":"https://api.github.com/users/Puriney/events{/privacy}","received_events_url":"https://api.github.com/users/Puriney/received_events","type":"User","site_admin":false},"body":"**How**\r\n\r\n\r\nThe Xavier initializer is borrowed from python implementation. This PR enables R package support `factor_type = c('in', 'out', 'avg')` and `rnd_type = c('gaussian', 'uniform')` to generate weights.\r\n\r\n`clip_gradient` is implemented via `pmax`/`pmin` R built-in function. \r\n\r\n`lr_scheduler` is referring to python implementation `python/lr_scheduler.py`.  I noticed there is some discussion among your developer team and hopefully my PR makes it as well. If @thirdwing  could revise this PR, it will be appreciated ;-)\r\n\r\n**What**\r\n\r\nGive a try using the official example:\r\n\r\n```r\r\n# Using regerssion as example to test my xavier initializer\r\n\r\n# Data\r\ndata(BostonHousing, package=\"mlbench\")\r\ntrain.ind = seq(1, 506, 3)\r\ntrain.x = data.matrix(BostonHousing[train.ind, -14])\r\ntrain.y = BostonHousing[train.ind, 14]\r\ntest.x = data.matrix(BostonHousing[-train.ind, -14])\r\ntest.y = BostonHousing[-train.ind, 14]\r\n\r\n# Computing Symbols\r\ndata <- mx.symbol.Variable(\"data\")\r\nfc1 <- mx.symbol.FullyConnected(data, num_hidden=1)\r\nlro <- mx.symbol.LinearRegressionOutput(fc1)\r\n\r\n# Training model\r\nprint(\"Default training\")\r\nmx.set.seed(0)\r\nmodel <-\r\n  mx.model.FeedForward.create(lro, X=train.x, y=train.y,\r\n                              ctx=mx.cpu(), num.round=50, array.batch.size=20,\r\n                              learning.rate=2e-6, momentum=0.9,\r\n                              eval.metric=mx.metric.rmse)\r\n\r\n\r\n# Prediction using test data\r\npreds <- predict(model, test.x)\r\n# See diff\r\ndiff <- sqrt(mean((preds-test.y)^2))\r\n\r\n# Training model with Xavier\r\nprint(\"Traing with Xavier initializer\")\r\nmx.set.seed(0)\r\nmodelX <-\r\n  mx.model.FeedForward.create(lro, X=train.x, y=train.y,\r\n                              ctx=mx.cpu(), num.round=50, array.batch.size=20,\r\n                              learning.rate=2e-6, momentum=0.9,\r\n                              eval.metric=mx.metric.rmse,\r\n                              initializer = mx.init.Xavier(\r\n                                rnd_type = \"gaussian\",\r\n                                factor_type = \"in\",\r\n                                magnitude = 2))\r\npredsX <- predict(modelX, test.x)\r\ndiffX <- sqrt(mean((predsX - test.y)^2))\r\n\r\n# Training model with clipping on gradient\r\nprint(\"Training model with clip\")\r\nmx.set.seed(0)\r\nmodelC <-\r\n  mx.model.FeedForward.create(lro, X=train.x, y=train.y,\r\n                              ctx=mx.cpu(), num.round=50, array.batch.size=20,\r\n                              learning.rate=2e-6, momentum=0.9,\r\n                              eval.metric=mx.metric.rmse,\r\n                              clip_gradient = 1)\r\npredsC <- predict(modelC, test.x)\r\ndiffC <- sqrt(mean((predsC - test.y)^2))\r\n\r\n# Training model with clipping on gradient\r\nprint(\"Training model with schedule\")\r\nmx.set.seed(0)\r\nmodelS <-\r\n  mx.model.FeedForward.create(lro, X=train.x, y=train.y,\r\n                              ctx=mx.cpu(), num.round=50, array.batch.size=20,\r\n                              learning.rate=2e-6, momentum=0.9,\r\n                              eval.metric=mx.metric.rmse,\r\n                              lr_scheduler=FactorScheduler(10, 0.9))\r\n\r\npredsS <- predict(modelS, test.x)\r\ndiffS <- sqrt(mean((predsS - test.y)^2))\r\n\r\nprint(diff)\r\nprint(diffX)\r\nprint(diffC)\r\nprint(diffS)\r\n```\r\n\r\n```\r\n[1] 7.800502\r\n[1] 7.888685\r\n[1] 16.15398\r\n[1] 8.324275\r\n```\r\n\r\nCc: @thirdwing  as I am interested in participating your GSOC 2016 project.","created_at":"2016-03-02T02:18:02Z","updated_at":"2016-03-03T22:09:43Z","closed_at":"2016-03-03T22:09:43Z","merged_at":"2016-03-03T22:09:43Z","merge_commit_sha":"8214e54b1f41aa30f34a538ea5ec8ef2ae48eac3","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/dmlc/mxnet/pulls/1554/commits","review_comments_url":"https://api.github.com/repos/dmlc/mxnet/pulls/1554/comments","review_comment_url":"https://api.github.com/repos/dmlc/mxnet/pulls/comments{/number}","comments_url":"https://api.github.com/repos/dmlc/mxnet/issues/1554/comments","statuses_url":"https://api.github.com/repos/dmlc/mxnet/statuses/3c5348832b632efa461c16720ee7103bd06df1c5","head":{"label":"Puriney:master","ref":"master","sha":"3c5348832b632efa461c16720ee7103bd06df1c5","user":{"login":"Puriney","id":1233372,"avatar_url":"https://avatars.githubusercontent.com/u/1233372?v=3","gravatar_id":"","url":"https://api.github.com/users/Puriney","html_url":"https://github.com/Puriney","followers_url":"https://api.github.com/users/Puriney/followers","following_url":"https://api.github.com/users/Puriney/following{/other_user}","gists_url":"https://api.github.com/users/Puriney/gists{/gist_id}","starred_url":"https://api.github.com/users/Puriney/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Puriney/subscriptions","organizations_url":"https://api.github.com/users/Puriney/orgs","repos_url":"https://api.github.com/users/Puriney/repos","events_url":"https://api.github.com/users/Puriney/events{/privacy}","received_events_url":"https://api.github.com/users/Puriney/received_events","type":"User","site_admin":false},"repo":{"id":52924699,"name":"mxnet","full_name":"Puriney/mxnet","owner":{"login":"Puriney","id":1233372,"avatar_url":"https://avatars.githubusercontent.com/u/1233372?v=3","gravatar_id":"","url":"https://api.github.com/users/Puriney","html_url":"https://github.com/Puriney","followers_url":"https://api.github.com/users/Puriney/followers","following_url":"https://api.github.com/users/Puriney/following{/other_user}","gists_url":"https://api.github.com/users/Puriney/gists{/gist_id}","starred_url":"https://api.github.com/users/Puriney/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Puriney/subscriptions","organizations_url":"https://api.github.com/users/Puriney/orgs","repos_url":"https://api.github.com/users/Puriney/repos","events_url":"https://api.github.com/users/Puriney/events{/privacy}","received_events_url":"https://api.github.com/users/Puriney/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/Puriney/mxnet","description":"Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Go, Javascript and more","fork":true,"url":"https://api.github.com/repos/Puriney/mxnet","forks_url":"https://api.github.com/repos/Puriney/mxnet/forks","keys_url":"https://api.github.com/repos/Puriney/mxnet/keys{/key_id}","collaborators_url":"https://api.github.com/repos/Puriney/mxnet/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/Puriney/mxnet/teams","hooks_url":"https://api.github.com/repos/Puriney/mxnet/hooks","issue_events_url":"https://api.github.com/repos/Puriney/mxnet/issues/events{/number}","events_url":"https://api.github.com/repos/Puriney/mxnet/events","assignees_url":"https://api.github.com/repos/Puriney/mxnet/assignees{/user}","branches_url":"https://api.github.com/repos/Puriney/mxnet/branches{/branch}","tags_url":"https://api.github.com/repos/Puriney/mxnet/tags","blobs_url":"https://api.github.com/repos/Puriney/mxnet/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/Puriney/mxnet/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/Puriney/mxnet/git/refs{/sha}","trees_url":"https://api.github.com/repos/Puriney/mxnet/git/trees{/sha}","statuses_url":"https://api.github.com/repos/Puriney/mxnet/statuses/{sha}","languages_url":"https://api.github.com/repos/Puriney/mxnet/languages","stargazers_url":"https://api.github.com/repos/Puriney/mxnet/stargazers","contributors_url":"https://api.github.com/repos/Puriney/mxnet/contributors","subscribers_url":"https://api.github.com/repos/Puriney/mxnet/subscribers","subscription_url":"https://api.github.com/repos/Puriney/mxnet/subscription","commits_url":"https://api.github.com/repos/Puriney/mxnet/commits{/sha}","git_commits_url":"https://api.github.com/repos/Puriney/mxnet/git/commits{/sha}","comments_url":"https://api.github.com/repos/Puriney/mxnet/comments{/number}","issue_comment_url":"https://api.github.com/repos/Puriney/mxnet/issues/comments{/number}","contents_url":"https://api.github.com/repos/Puriney/mxnet/contents/{+path}","compare_url":"https://api.github.com/repos/Puriney/mxnet/compare/{base}...{head}","merges_url":"https://api.github.com/repos/Puriney/mxnet/merges","archive_url":"https://api.github.com/repos/Puriney/mxnet/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/Puriney/mxnet/downloads","issues_url":"https://api.github.com/repos/Puriney/mxnet/issues{/number}","pulls_url":"https://api.github.com/repos/Puriney/mxnet/pulls{/number}","milestones_url":"https://api.github.com/repos/Puriney/mxnet/milestones{/number}","notifications_url":"https://api.github.com/repos/Puriney/mxnet/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/Puriney/mxnet/labels{/name}","releases_url":"https://api.github.com/repos/Puriney/mxnet/releases{/id}","deployments_url":"https://api.github.com/repos/Puriney/mxnet/deployments","created_at":"2016-03-02T01:50:13Z","updated_at":"2016-03-02T01:50:15Z","pushed_at":"2016-03-03T20:53:38Z","git_url":"git://github.com/Puriney/mxnet.git","ssh_url":"git@github.com:Puriney/mxnet.git","clone_url":"https://github.com/Puriney/mxnet.git","svn_url":"https://github.com/Puriney/mxnet","homepage":"http://mxnet.rtfd.org","size":4986,"stargazers_count":0,"watchers_count":0,"language":"C++","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"dmlc:master","ref":"master","sha":"6c37230a269be0988f3d56df2d1cf85f97722242","user":{"login":"dmlc","id":11508361,"avatar_url":"https://avatars.githubusercontent.com/u/11508361?v=3","gravatar_id":"","url":"https://api.github.com/users/dmlc","html_url":"https://github.com/dmlc","followers_url":"https://api.github.com/users/dmlc/followers","following_url":"https://api.github.com/users/dmlc/following{/other_user}","gists_url":"https://api.github.com/users/dmlc/gists{/gist_id}","starred_url":"https://api.github.com/users/dmlc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dmlc/subscriptions","organizations_url":"https://api.github.com/users/dmlc/orgs","repos_url":"https://api.github.com/users/dmlc/repos","events_url":"https://api.github.com/users/dmlc/events{/privacy}","received_events_url":"https://api.github.com/users/dmlc/received_events","type":"Organization","site_admin":false},"repo":{"id":34864402,"name":"mxnet","full_name":"dmlc/mxnet","owner":{"login":"dmlc","id":11508361,"avatar_url":"https://avatars.githubusercontent.com/u/11508361?v=3","gravatar_id":"","url":"https://api.github.com/users/dmlc","html_url":"https://github.com/dmlc","followers_url":"https://api.github.com/users/dmlc/followers","following_url":"https://api.github.com/users/dmlc/following{/other_user}","gists_url":"https://api.github.com/users/dmlc/gists{/gist_id}","starred_url":"https://api.github.com/users/dmlc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dmlc/subscriptions","organizations_url":"https://api.github.com/users/dmlc/orgs","repos_url":"https://api.github.com/users/dmlc/repos","events_url":"https://api.github.com/users/dmlc/events{/privacy}","received_events_url":"https://api.github.com/users/dmlc/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/dmlc/mxnet","description":"Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Go, Javascript and more","fork":false,"url":"https://api.github.com/repos/dmlc/mxnet","forks_url":"https://api.github.com/repos/dmlc/mxnet/forks","keys_url":"https://api.github.com/repos/dmlc/mxnet/keys{/key_id}","collaborators_url":"https://api.github.com/repos/dmlc/mxnet/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/dmlc/mxnet/teams","hooks_url":"https://api.github.com/repos/dmlc/mxnet/hooks","issue_events_url":"https://api.github.com/repos/dmlc/mxnet/issues/events{/number}","events_url":"https://api.github.com/repos/dmlc/mxnet/events","assignees_url":"https://api.github.com/repos/dmlc/mxnet/assignees{/user}","branches_url":"https://api.github.com/repos/dmlc/mxnet/branches{/branch}","tags_url":"https://api.github.com/repos/dmlc/mxnet/tags","blobs_url":"https://api.github.com/repos/dmlc/mxnet/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/dmlc/mxnet/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/dmlc/mxnet/git/refs{/sha}","trees_url":"https://api.github.com/repos/dmlc/mxnet/git/trees{/sha}","statuses_url":"https://api.github.com/repos/dmlc/mxnet/statuses/{sha}","languages_url":"https://api.github.com/repos/dmlc/mxnet/languages","stargazers_url":"https://api.github.com/repos/dmlc/mxnet/stargazers","contributors_url":"https://api.github.com/repos/dmlc/mxnet/contributors","subscribers_url":"https://api.github.com/repos/dmlc/mxnet/subscribers","subscription_url":"https://api.github.com/repos/dmlc/mxnet/subscription","commits_url":"https://api.github.com/repos/dmlc/mxnet/commits{/sha}","git_commits_url":"https://api.github.com/repos/dmlc/mxnet/git/commits{/sha}","comments_url":"https://api.github.com/repos/dmlc/mxnet/comments{/number}","issue_comment_url":"https://api.github.com/repos/dmlc/mxnet/issues/comments{/number}","contents_url":"https://api.github.com/repos/dmlc/mxnet/contents/{+path}","compare_url":"https://api.github.com/repos/dmlc/mxnet/compare/{base}...{head}","merges_url":"https://api.github.com/repos/dmlc/mxnet/merges","archive_url":"https://api.github.com/repos/dmlc/mxnet/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/dmlc/mxnet/downloads","issues_url":"https://api.github.com/repos/dmlc/mxnet/issues{/number}","pulls_url":"https://api.github.com/repos/dmlc/mxnet/pulls{/number}","milestones_url":"https://api.github.com/repos/dmlc/mxnet/milestones{/number}","notifications_url":"https://api.github.com/repos/dmlc/mxnet/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/dmlc/mxnet/labels{/name}","releases_url":"https://api.github.com/repos/dmlc/mxnet/releases{/id}","deployments_url":"https://api.github.com/repos/dmlc/mxnet/deployments","created_at":"2015-04-30T16:21:15Z","updated_at":"2016-03-03T20:45:40Z","pushed_at":"2016-03-03T22:09:43Z","git_url":"git://github.com/dmlc/mxnet.git","ssh_url":"git@github.com:dmlc/mxnet.git","clone_url":"https://github.com/dmlc/mxnet.git","svn_url":"https://github.com/dmlc/mxnet","homepage":"http://mxnet.rtfd.org","size":6898,"stargazers_count":2783,"watchers_count":2783,"language":"C++","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":942,"mirror_url":null,"open_issues_count":232,"forks":942,"open_issues":232,"watchers":2783,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/dmlc/mxnet/pulls/1554"},"html":{"href":"https://github.com/dmlc/mxnet/pull/1554"},"issue":{"href":"https://api.github.com/repos/dmlc/mxnet/issues/1554"},"comments":{"href":"https://api.github.com/repos/dmlc/mxnet/issues/1554/comments"},"review_comments":{"href":"https://api.github.com/repos/dmlc/mxnet/pulls/1554/comments"},"review_comment":{"href":"https://api.github.com/repos/dmlc/mxnet/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/dmlc/mxnet/pulls/1554/commits"},"statuses":{"href":"https://api.github.com/repos/dmlc/mxnet/statuses/3c5348832b632efa461c16720ee7103bd06df1c5"}},"merged":true,"mergeable":null,"mergeable_state":"unknown","merged_by":{"login":"thirdwing","id":1547093,"avatar_url":"https://avatars.githubusercontent.com/u/1547093?v=3","gravatar_id":"","url":"https://api.github.com/users/thirdwing","html_url":"https://github.com/thirdwing","followers_url":"https://api.github.com/users/thirdwing/followers","following_url":"https://api.github.com/users/thirdwing/following{/other_user}","gists_url":"https://api.github.com/users/thirdwing/gists{/gist_id}","starred_url":"https://api.github.com/users/thirdwing/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/thirdwing/subscriptions","organizations_url":"https://api.github.com/users/thirdwing/orgs","repos_url":"https://api.github.com/users/thirdwing/repos","events_url":"https://api.github.com/users/thirdwing/events{/privacy}","received_events_url":"https://api.github.com/users/thirdwing/received_events","type":"User","site_admin":false},"comments":3,"review_comments":1,"commits":2,"additions":127,"deletions":4,"changed_files":7}}