{"action":"created","comment":{"url":"https://api.github.com/repos/mlpack/mlpack/pulls/comments/272809073","pull_request_review_id":223556047,"id":272809073,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDI3MjgwOTA3Mw==","diff_hunk":"@@ -0,0 +1,87 @@\n+/**\n+ * @file c_relu_impl.hpp\n+ * @author Jeffin Sam\n+ *\n+ * Implementation of CReLU layer.\n+ * Introduced by,\n+ * Wenling Shang, Kihyuk Sohn, Diogo Almeida, Honglak Lee,\n+ * \"https://arxiv.org/abs/1603.05201\", 16th March 2016.\n+ *\n+ * mlpack is free software; you may redistribute it and/or modify it under the\n+ * terms of the 3-clause BSD license.  You should have received a copy of the\n+ * 3-clause BSD license along with mlpack.  If not, see\n+ * http://www.opensource.org/licenses/BSD-3-Clause for more information.\n+ */\n+#ifndef MLPACK_METHODS_ANN_LAYER_C_RELU_IMPL_HPP\n+#define MLPACK_METHODS_ANN_LAYER_C_RELU_IMPL_HPP\n+\n+// In case it hasn't yet been included.\n+#include \"c_relu.hpp\"\n+\n+\n+namespace mlpack {\n+namespace ann /** Artificial Neural Network. */ {\n+\n+template<typename InputDataType, typename OutputDataType>\n+CReLU<InputDataType, OutputDataType>::CReLU()\n+{\n+  // Nothing to do here.\n+}\n+\n+template<typename InputDataType, typename OutputDataType>\n+template<typename InputType, typename OutputType>\n+void CReLU<InputDataType, OutputDataType>::Forward(\n+    const InputType&& input, OutputType&& output)\n+{\n+  // Optimisation needed\n+  OutputType temp1;\n+  OutputType temp2;\n+  Fn(input, temp1);\n+  InputType inptemp=-1*input;\n+  Fn(inptemp, temp2);\n+  // Concat Neg and Pos Relu\n+  output = arma::join_cols(temp1,temp2);\n+}\n+\n+template<typename InputDataType, typename OutputDataType>\n+template<typename DataType>\n+void CReLU<InputDataType, OutputDataType>::Backward(\n+    const DataType&& input, DataType&& gy, DataType&& g)\n+{\n+  DataType derivative;\n+  Deriv(input,derivative);\n+  DataType temp;\n+  temp = gy % derivative;\n+  g= temp.rows(0, (input.n_rows/2-1) )-temp.rows(input.n_rows/2,(input.n_rows-1));","path":"src/mlpack/methods/ann/layer/c_relu_impl.hpp","position":55,"original_position":55,"commit_id":"22eb87dd25721d6bb41be1221303ef342b8c1077","original_commit_id":"0d216daf79292d11908b563c892e4559d01bf780","user":{"login":"jeffin143","id":21286134,"node_id":"MDQ6VXNlcjIxMjg2MTM0","avatar_url":"https://avatars3.githubusercontent.com/u/21286134?v=4","gravatar_id":"","url":"https://api.github.com/users/jeffin143","html_url":"https://github.com/jeffin143","followers_url":"https://api.github.com/users/jeffin143/followers","following_url":"https://api.github.com/users/jeffin143/following{/other_user}","gists_url":"https://api.github.com/users/jeffin143/gists{/gist_id}","starred_url":"https://api.github.com/users/jeffin143/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jeffin143/subscriptions","organizations_url":"https://api.github.com/users/jeffin143/orgs","repos_url":"https://api.github.com/users/jeffin143/repos","events_url":"https://api.github.com/users/jeffin143/events{/privacy}","received_events_url":"https://api.github.com/users/jeffin143/received_events","type":"User","site_admin":false},"body":"Backward Propagation, Since the CRELU double the input and hence which backward propagation `w^T*g` throws error due to size miss match, so making it of same size","created_at":"2019-04-06T20:48:39Z","updated_at":"2019-04-06T20:48:39Z","html_url":"https://github.com/mlpack/mlpack/pull/1843#discussion_r272809073","pull_request_url":"https://api.github.com/repos/mlpack/mlpack/pulls/1843","author_association":"CONTRIBUTOR","_links":{"self":{"href":"https://api.github.com/repos/mlpack/mlpack/pulls/comments/272809073"},"html":{"href":"https://github.com/mlpack/mlpack/pull/1843#discussion_r272809073"},"pull_request":{"href":"https://api.github.com/repos/mlpack/mlpack/pulls/1843"}}},"pull_request":{"url":"https://api.github.com/repos/mlpack/mlpack/pulls/1843","id":268061491,"node_id":"MDExOlB1bGxSZXF1ZXN0MjY4MDYxNDkx","html_url":"https://github.com/mlpack/mlpack/pull/1843","diff_url":"https://github.com/mlpack/mlpack/pull/1843.diff","patch_url":"https://github.com/mlpack/mlpack/pull/1843.patch","issue_url":"https://api.github.com/repos/mlpack/mlpack/issues/1843","number":1843,"state":"open","locked":false,"title":"Implemetation of Concatenated ReLU Activation","user":{"login":"jeffin143","id":21286134,"node_id":"MDQ6VXNlcjIxMjg2MTM0","avatar_url":"https://avatars3.githubusercontent.com/u/21286134?v=4","gravatar_id":"","url":"https://api.github.com/users/jeffin143","html_url":"https://github.com/jeffin143","followers_url":"https://api.github.com/users/jeffin143/followers","following_url":"https://api.github.com/users/jeffin143/following{/other_user}","gists_url":"https://api.github.com/users/jeffin143/gists{/gist_id}","starred_url":"https://api.github.com/users/jeffin143/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jeffin143/subscriptions","organizations_url":"https://api.github.com/users/jeffin143/orgs","repos_url":"https://api.github.com/users/jeffin143/repos","events_url":"https://api.github.com/users/jeffin143/events{/privacy}","received_events_url":"https://api.github.com/users/jeffin143/received_events","type":"User","site_admin":false},"body":"I tried implementing CRELU Activation, following the research paper by `Wenling Shang, Kihyuk Sohn, Diogo Almeida, Honglak Lee,` in [2016](https://arxiv.org/abs/1603.05201)\r\n\r\nThis activation scheme preserves both positive and negative phase information while enforcing non-saturated  non-linearity.The unique nature  of CReLU allows a mathematical characterization of convolution layers in terms of reconstruction property.\r\n\r\nCReLU has shown to perform very well in Convolutional Neural Networks.CReLU naturally reduces redundancy of learning separate filters that differ in phase only.\r\n\r\nI encountered lot of doubts during implementation and have done some tweaking during implementation.So it may be not be fully sound on implementation part. So i won't mind even if takes long for you to review this PR, But i would like to get your input on the code.","created_at":"2019-04-06T20:40:32Z","updated_at":"2019-04-06T20:48:39Z","closed_at":null,"merged_at":null,"merge_commit_sha":"d89d1125e3977493a908d5ab22fba5fef9627648","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[{"id":1200134486,"node_id":"MDU6TGFiZWwxMjAwMTM0NDg2","url":"https://api.github.com/repos/mlpack/mlpack/labels/s:%20needs%20review","name":"s: needs review","color":"ffff55","default":false},{"id":1200136603,"node_id":"MDU6TGFiZWwxMjAwMTM2NjAz","url":"https://api.github.com/repos/mlpack/mlpack/labels/s:%20unanswered","name":"s: unanswered","color":"ffff55","default":false},{"id":1200223274,"node_id":"MDU6TGFiZWwxMjAwMjIzMjc0","url":"https://api.github.com/repos/mlpack/mlpack/labels/s:%20unlabeled","name":"s: unlabeled","color":"ffff55","default":false}],"milestone":null,"commits_url":"https://api.github.com/repos/mlpack/mlpack/pulls/1843/commits","review_comments_url":"https://api.github.com/repos/mlpack/mlpack/pulls/1843/comments","review_comment_url":"https://api.github.com/repos/mlpack/mlpack/pulls/comments{/number}","comments_url":"https://api.github.com/repos/mlpack/mlpack/issues/1843/comments","statuses_url":"https://api.github.com/repos/mlpack/mlpack/statuses/22eb87dd25721d6bb41be1221303ef342b8c1077","head":{"label":"jeffin143:crelu","ref":"crelu","sha":"22eb87dd25721d6bb41be1221303ef342b8c1077","user":{"login":"jeffin143","id":21286134,"node_id":"MDQ6VXNlcjIxMjg2MTM0","avatar_url":"https://avatars3.githubusercontent.com/u/21286134?v=4","gravatar_id":"","url":"https://api.github.com/users/jeffin143","html_url":"https://github.com/jeffin143","followers_url":"https://api.github.com/users/jeffin143/followers","following_url":"https://api.github.com/users/jeffin143/following{/other_user}","gists_url":"https://api.github.com/users/jeffin143/gists{/gist_id}","starred_url":"https://api.github.com/users/jeffin143/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jeffin143/subscriptions","organizations_url":"https://api.github.com/users/jeffin143/orgs","repos_url":"https://api.github.com/users/jeffin143/repos","events_url":"https://api.github.com/users/jeffin143/events{/privacy}","received_events_url":"https://api.github.com/users/jeffin143/received_events","type":"User","site_admin":false},"repo":{"id":175831594,"node_id":"MDEwOlJlcG9zaXRvcnkxNzU4MzE1OTQ=","name":"mlpack","full_name":"jeffin143/mlpack","private":false,"owner":{"login":"jeffin143","id":21286134,"node_id":"MDQ6VXNlcjIxMjg2MTM0","avatar_url":"https://avatars3.githubusercontent.com/u/21286134?v=4","gravatar_id":"","url":"https://api.github.com/users/jeffin143","html_url":"https://github.com/jeffin143","followers_url":"https://api.github.com/users/jeffin143/followers","following_url":"https://api.github.com/users/jeffin143/following{/other_user}","gists_url":"https://api.github.com/users/jeffin143/gists{/gist_id}","starred_url":"https://api.github.com/users/jeffin143/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jeffin143/subscriptions","organizations_url":"https://api.github.com/users/jeffin143/orgs","repos_url":"https://api.github.com/users/jeffin143/repos","events_url":"https://api.github.com/users/jeffin143/events{/privacy}","received_events_url":"https://api.github.com/users/jeffin143/received_events","type":"User","site_admin":false},"html_url":"https://github.com/jeffin143/mlpack","description":"mlpack: a scalable C++ machine learning library -- ","fork":true,"url":"https://api.github.com/repos/jeffin143/mlpack","forks_url":"https://api.github.com/repos/jeffin143/mlpack/forks","keys_url":"https://api.github.com/repos/jeffin143/mlpack/keys{/key_id}","collaborators_url":"https://api.github.com/repos/jeffin143/mlpack/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/jeffin143/mlpack/teams","hooks_url":"https://api.github.com/repos/jeffin143/mlpack/hooks","issue_events_url":"https://api.github.com/repos/jeffin143/mlpack/issues/events{/number}","events_url":"https://api.github.com/repos/jeffin143/mlpack/events","assignees_url":"https://api.github.com/repos/jeffin143/mlpack/assignees{/user}","branches_url":"https://api.github.com/repos/jeffin143/mlpack/branches{/branch}","tags_url":"https://api.github.com/repos/jeffin143/mlpack/tags","blobs_url":"https://api.github.com/repos/jeffin143/mlpack/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/jeffin143/mlpack/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/jeffin143/mlpack/git/refs{/sha}","trees_url":"https://api.github.com/repos/jeffin143/mlpack/git/trees{/sha}","statuses_url":"https://api.github.com/repos/jeffin143/mlpack/statuses/{sha}","languages_url":"https://api.github.com/repos/jeffin143/mlpack/languages","stargazers_url":"https://api.github.com/repos/jeffin143/mlpack/stargazers","contributors_url":"https://api.github.com/repos/jeffin143/mlpack/contributors","subscribers_url":"https://api.github.com/repos/jeffin143/mlpack/subscribers","subscription_url":"https://api.github.com/repos/jeffin143/mlpack/subscription","commits_url":"https://api.github.com/repos/jeffin143/mlpack/commits{/sha}","git_commits_url":"https://api.github.com/repos/jeffin143/mlpack/git/commits{/sha}","comments_url":"https://api.github.com/repos/jeffin143/mlpack/comments{/number}","issue_comment_url":"https://api.github.com/repos/jeffin143/mlpack/issues/comments{/number}","contents_url":"https://api.github.com/repos/jeffin143/mlpack/contents/{+path}","compare_url":"https://api.github.com/repos/jeffin143/mlpack/compare/{base}...{head}","merges_url":"https://api.github.com/repos/jeffin143/mlpack/merges","archive_url":"https://api.github.com/repos/jeffin143/mlpack/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/jeffin143/mlpack/downloads","issues_url":"https://api.github.com/repos/jeffin143/mlpack/issues{/number}","pulls_url":"https://api.github.com/repos/jeffin143/mlpack/pulls{/number}","milestones_url":"https://api.github.com/repos/jeffin143/mlpack/milestones{/number}","notifications_url":"https://api.github.com/repos/jeffin143/mlpack/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/jeffin143/mlpack/labels{/name}","releases_url":"https://api.github.com/repos/jeffin143/mlpack/releases{/id}","deployments_url":"https://api.github.com/repos/jeffin143/mlpack/deployments","created_at":"2019-03-15T14:05:21Z","updated_at":"2019-03-27T10:36:26Z","pushed_at":"2019-04-06T20:44:22Z","git_url":"git://github.com/jeffin143/mlpack.git","ssh_url":"git@github.com:jeffin143/mlpack.git","clone_url":"https://github.com/jeffin143/mlpack.git","svn_url":"https://github.com/jeffin143/mlpack","homepage":"http://www.mlpack.org/","size":246497,"stargazers_count":0,"watchers_count":0,"language":"C++","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":0,"license":{"key":"other","name":"Other","spdx_id":"NOASSERTION","url":null,"node_id":"MDc6TGljZW5zZTA="},"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"mlpack:master","ref":"master","sha":"628de7613adb6e7075a67f07ddf11fe86cef139b","user":{"login":"mlpack","id":10216045,"node_id":"MDEyOk9yZ2FuaXphdGlvbjEwMjE2MDQ1","avatar_url":"https://avatars0.githubusercontent.com/u/10216045?v=4","gravatar_id":"","url":"https://api.github.com/users/mlpack","html_url":"https://github.com/mlpack","followers_url":"https://api.github.com/users/mlpack/followers","following_url":"https://api.github.com/users/mlpack/following{/other_user}","gists_url":"https://api.github.com/users/mlpack/gists{/gist_id}","starred_url":"https://api.github.com/users/mlpack/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mlpack/subscriptions","organizations_url":"https://api.github.com/users/mlpack/orgs","repos_url":"https://api.github.com/users/mlpack/repos","events_url":"https://api.github.com/users/mlpack/events{/privacy}","received_events_url":"https://api.github.com/users/mlpack/received_events","type":"Organization","site_admin":false},"repo":{"id":28149409,"node_id":"MDEwOlJlcG9zaXRvcnkyODE0OTQwOQ==","name":"mlpack","full_name":"mlpack/mlpack","private":false,"owner":{"login":"mlpack","id":10216045,"node_id":"MDEyOk9yZ2FuaXphdGlvbjEwMjE2MDQ1","avatar_url":"https://avatars0.githubusercontent.com/u/10216045?v=4","gravatar_id":"","url":"https://api.github.com/users/mlpack","html_url":"https://github.com/mlpack","followers_url":"https://api.github.com/users/mlpack/followers","following_url":"https://api.github.com/users/mlpack/following{/other_user}","gists_url":"https://api.github.com/users/mlpack/gists{/gist_id}","starred_url":"https://api.github.com/users/mlpack/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mlpack/subscriptions","organizations_url":"https://api.github.com/users/mlpack/orgs","repos_url":"https://api.github.com/users/mlpack/repos","events_url":"https://api.github.com/users/mlpack/events{/privacy}","received_events_url":"https://api.github.com/users/mlpack/received_events","type":"Organization","site_admin":false},"html_url":"https://github.com/mlpack/mlpack","description":"mlpack: a scalable C++ machine learning library -- ","fork":false,"url":"https://api.github.com/repos/mlpack/mlpack","forks_url":"https://api.github.com/repos/mlpack/mlpack/forks","keys_url":"https://api.github.com/repos/mlpack/mlpack/keys{/key_id}","collaborators_url":"https://api.github.com/repos/mlpack/mlpack/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/mlpack/mlpack/teams","hooks_url":"https://api.github.com/repos/mlpack/mlpack/hooks","issue_events_url":"https://api.github.com/repos/mlpack/mlpack/issues/events{/number}","events_url":"https://api.github.com/repos/mlpack/mlpack/events","assignees_url":"https://api.github.com/repos/mlpack/mlpack/assignees{/user}","branches_url":"https://api.github.com/repos/mlpack/mlpack/branches{/branch}","tags_url":"https://api.github.com/repos/mlpack/mlpack/tags","blobs_url":"https://api.github.com/repos/mlpack/mlpack/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/mlpack/mlpack/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/mlpack/mlpack/git/refs{/sha}","trees_url":"https://api.github.com/repos/mlpack/mlpack/git/trees{/sha}","statuses_url":"https://api.github.com/repos/mlpack/mlpack/statuses/{sha}","languages_url":"https://api.github.com/repos/mlpack/mlpack/languages","stargazers_url":"https://api.github.com/repos/mlpack/mlpack/stargazers","contributors_url":"https://api.github.com/repos/mlpack/mlpack/contributors","subscribers_url":"https://api.github.com/repos/mlpack/mlpack/subscribers","subscription_url":"https://api.github.com/repos/mlpack/mlpack/subscription","commits_url":"https://api.github.com/repos/mlpack/mlpack/commits{/sha}","git_commits_url":"https://api.github.com/repos/mlpack/mlpack/git/commits{/sha}","comments_url":"https://api.github.com/repos/mlpack/mlpack/comments{/number}","issue_comment_url":"https://api.github.com/repos/mlpack/mlpack/issues/comments{/number}","contents_url":"https://api.github.com/repos/mlpack/mlpack/contents/{+path}","compare_url":"https://api.github.com/repos/mlpack/mlpack/compare/{base}...{head}","merges_url":"https://api.github.com/repos/mlpack/mlpack/merges","archive_url":"https://api.github.com/repos/mlpack/mlpack/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/mlpack/mlpack/downloads","issues_url":"https://api.github.com/repos/mlpack/mlpack/issues{/number}","pulls_url":"https://api.github.com/repos/mlpack/mlpack/pulls{/number}","milestones_url":"https://api.github.com/repos/mlpack/mlpack/milestones{/number}","notifications_url":"https://api.github.com/repos/mlpack/mlpack/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/mlpack/mlpack/labels{/name}","releases_url":"https://api.github.com/repos/mlpack/mlpack/releases{/id}","deployments_url":"https://api.github.com/repos/mlpack/mlpack/deployments","created_at":"2014-12-17T18:16:59Z","updated_at":"2019-04-06T12:57:12Z","pushed_at":"2019-04-06T20:44:24Z","git_url":"git://github.com/mlpack/mlpack.git","ssh_url":"git@github.com:mlpack/mlpack.git","clone_url":"https://github.com/mlpack/mlpack.git","svn_url":"https://github.com/mlpack/mlpack","homepage":"http://www.mlpack.org/","size":246475,"stargazers_count":2753,"watchers_count":2753,"language":"C++","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":1008,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":132,"license":{"key":"other","name":"Other","spdx_id":"NOASSERTION","url":null,"node_id":"MDc6TGljZW5zZTA="},"forks":1008,"open_issues":132,"watchers":2753,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/mlpack/mlpack/pulls/1843"},"html":{"href":"https://github.com/mlpack/mlpack/pull/1843"},"issue":{"href":"https://api.github.com/repos/mlpack/mlpack/issues/1843"},"comments":{"href":"https://api.github.com/repos/mlpack/mlpack/issues/1843/comments"},"review_comments":{"href":"https://api.github.com/repos/mlpack/mlpack/pulls/1843/comments"},"review_comment":{"href":"https://api.github.com/repos/mlpack/mlpack/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/mlpack/mlpack/pulls/1843/commits"},"statuses":{"href":"https://api.github.com/repos/mlpack/mlpack/statuses/22eb87dd25721d6bb41be1221303ef342b8c1077"}},"author_association":"CONTRIBUTOR"}}
{"actor":{"display_login":"jeffin143"}}