{"action":"created","comment":{"url":"https://api.github.com/repos/LLNL/lbann/pulls/comments/196537936","pull_request_review_id":130110170,"id":196537936,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDE5NjUzNzkzNg==","diff_hunk":"@@ -136,7 +136,7 @@ class selu_dropout : public regularizer_layer {\n   void bp_compute() override {\n     if (this->m_model->get_execution_mode() != execution_mode::training\n         || m_keep_prob < 0.0f) {\n-      El::LockedView(get_error_signals(), get_prev_error_signals());\n+      El::Copy(get_prev_error_signals(), get_error_signals());","path":"include/lbann/layers/regularizers/selu_dropout.hpp","position":5,"original_position":5,"commit_id":"50b68b03255fdb1079435084fcbb2569d6e3547c","original_commit_id":"50b68b03255fdb1079435084fcbb2569d6e3547c","user":{"login":"timmoon10","id":4406448,"node_id":"MDQ6VXNlcjQ0MDY0NDg=","avatar_url":"https://avatars0.githubusercontent.com/u/4406448?v=4","gravatar_id":"","url":"https://api.github.com/users/timmoon10","html_url":"https://github.com/timmoon10","followers_url":"https://api.github.com/users/timmoon10/followers","following_url":"https://api.github.com/users/timmoon10/following{/other_user}","gists_url":"https://api.github.com/users/timmoon10/gists{/gist_id}","starred_url":"https://api.github.com/users/timmoon10/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/timmoon10/subscriptions","organizations_url":"https://api.github.com/users/timmoon10/orgs","repos_url":"https://api.github.com/users/timmoon10/repos","events_url":"https://api.github.com/users/timmoon10/events{/privacy}","received_events_url":"https://api.github.com/users/timmoon10/received_events","type":"User","site_admin":false},"body":"On second thought, it takes some slight convolutions to make the activations or error signals a view, even more when it changes between owning data during training and being a view during evaluation. The SeLU stuff is exotic and not yet performance critical (especially since this optimization would only speed up evaluation), so I think simplicity is more important.","created_at":"2018-06-19T18:46:07Z","updated_at":"2018-06-19T18:46:08Z","html_url":"https://github.com/LLNL/lbann/pull/435#discussion_r196537936","pull_request_url":"https://api.github.com/repos/LLNL/lbann/pulls/435","author_association":"COLLABORATOR","_links":{"self":{"href":"https://api.github.com/repos/LLNL/lbann/pulls/comments/196537936"},"html":{"href":"https://github.com/LLNL/lbann/pull/435#discussion_r196537936"},"pull_request":{"href":"https://api.github.com/repos/LLNL/lbann/pulls/435"}},"in_reply_to_id":195880013},"pull_request":{"url":"https://api.github.com/repos/LLNL/lbann/pulls/435","id":195228441,"node_id":"MDExOlB1bGxSZXF1ZXN0MTk1MjI4NDQx","html_url":"https://github.com/LLNL/lbann/pull/435","diff_url":"https://github.com/LLNL/lbann/pull/435.diff","patch_url":"https://github.com/LLNL/lbann/pull/435.patch","issue_url":"https://api.github.com/repos/LLNL/lbann/issues/435","number":435,"state":"open","locked":false,"title":"Overwrite error signals during backprop instead of adding to them","user":{"login":"timmoon10","id":4406448,"node_id":"MDQ6VXNlcjQ0MDY0NDg=","avatar_url":"https://avatars0.githubusercontent.com/u/4406448?v=4","gravatar_id":"","url":"https://api.github.com/users/timmoon10","html_url":"https://github.com/timmoon10","followers_url":"https://api.github.com/users/timmoon10/followers","following_url":"https://api.github.com/users/timmoon10/following{/other_user}","gists_url":"https://api.github.com/users/timmoon10/gists{/gist_id}","starred_url":"https://api.github.com/users/timmoon10/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/timmoon10/subscriptions","organizations_url":"https://api.github.com/users/timmoon10/orgs","repos_url":"https://api.github.com/users/timmoon10/repos","events_url":"https://api.github.com/users/timmoon10/events{/privacy}","received_events_url":"https://api.github.com/users/timmoon10/received_events","type":"User","site_admin":false},"body":"I see ~8% speedup in AlexNet on Surface gpgpu, with no significant change to learning behavior. Gradient checks on model_zoo/tests/model_mnist_conv_graph.prototext come back clean. This closes #347 and is a step toward #193.\r\n \r\nThe kludgiest part of this PR involves the target layer. The objective function terms assume that the target layer's error signals are zeroed out when we compute their gradients, so the target layer's forward prop clears out the error signals. I've also added a default implementation of backprop which passes zero error signals. This makes it easier to naively implement layers (computed gradients will be inaccurate but not catastrophic).\r\n\r\nNote that this breaks the VAE KL divergence objective function term, which is being removed in #431. It will probably also conflict with #405 and #417, so those should be merged first.","created_at":"2018-06-15T16:41:26Z","updated_at":"2018-06-19T18:46:08Z","closed_at":null,"merged_at":null,"merge_commit_sha":"45adbe063bfe44313304858076e6ffae61daf448","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[{"id":374636004,"node_id":"MDU6TGFiZWwzNzQ2MzYwMDQ=","url":"https://api.github.com/repos/LLNL/lbann/labels/enhancement","name":"enhancement","color":"84b6eb","default":true},{"id":808127779,"node_id":"MDU6TGFiZWw4MDgxMjc3Nzk=","url":"https://api.github.com/repos/LLNL/lbann/labels/review%20requested","name":"review requested","color":"a053e0","default":false}],"milestone":null,"commits_url":"https://api.github.com/repos/LLNL/lbann/pulls/435/commits","review_comments_url":"https://api.github.com/repos/LLNL/lbann/pulls/435/comments","review_comment_url":"https://api.github.com/repos/LLNL/lbann/pulls/comments{/number}","comments_url":"https://api.github.com/repos/LLNL/lbann/issues/435/comments","statuses_url":"https://api.github.com/repos/LLNL/lbann/statuses/50b68b03255fdb1079435084fcbb2569d6e3547c","head":{"label":"timmoon10:overwrite-error-signals-dev","ref":"overwrite-error-signals-dev","sha":"50b68b03255fdb1079435084fcbb2569d6e3547c","user":{"login":"timmoon10","id":4406448,"node_id":"MDQ6VXNlcjQ0MDY0NDg=","avatar_url":"https://avatars0.githubusercontent.com/u/4406448?v=4","gravatar_id":"","url":"https://api.github.com/users/timmoon10","html_url":"https://github.com/timmoon10","followers_url":"https://api.github.com/users/timmoon10/followers","following_url":"https://api.github.com/users/timmoon10/following{/other_user}","gists_url":"https://api.github.com/users/timmoon10/gists{/gist_id}","starred_url":"https://api.github.com/users/timmoon10/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/timmoon10/subscriptions","organizations_url":"https://api.github.com/users/timmoon10/orgs","repos_url":"https://api.github.com/users/timmoon10/repos","events_url":"https://api.github.com/users/timmoon10/events{/privacy}","received_events_url":"https://api.github.com/users/timmoon10/received_events","type":"User","site_admin":false},"repo":{"id":107584119,"node_id":"MDEwOlJlcG9zaXRvcnkxMDc1ODQxMTk=","name":"lbann","full_name":"timmoon10/lbann","owner":{"login":"timmoon10","id":4406448,"node_id":"MDQ6VXNlcjQ0MDY0NDg=","avatar_url":"https://avatars0.githubusercontent.com/u/4406448?v=4","gravatar_id":"","url":"https://api.github.com/users/timmoon10","html_url":"https://github.com/timmoon10","followers_url":"https://api.github.com/users/timmoon10/followers","following_url":"https://api.github.com/users/timmoon10/following{/other_user}","gists_url":"https://api.github.com/users/timmoon10/gists{/gist_id}","starred_url":"https://api.github.com/users/timmoon10/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/timmoon10/subscriptions","organizations_url":"https://api.github.com/users/timmoon10/orgs","repos_url":"https://api.github.com/users/timmoon10/repos","events_url":"https://api.github.com/users/timmoon10/events{/privacy}","received_events_url":"https://api.github.com/users/timmoon10/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/timmoon10/lbann","description":"Livermore Big Artificial Neural Network Toolkit","fork":true,"url":"https://api.github.com/repos/timmoon10/lbann","forks_url":"https://api.github.com/repos/timmoon10/lbann/forks","keys_url":"https://api.github.com/repos/timmoon10/lbann/keys{/key_id}","collaborators_url":"https://api.github.com/repos/timmoon10/lbann/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/timmoon10/lbann/teams","hooks_url":"https://api.github.com/repos/timmoon10/lbann/hooks","issue_events_url":"https://api.github.com/repos/timmoon10/lbann/issues/events{/number}","events_url":"https://api.github.com/repos/timmoon10/lbann/events","assignees_url":"https://api.github.com/repos/timmoon10/lbann/assignees{/user}","branches_url":"https://api.github.com/repos/timmoon10/lbann/branches{/branch}","tags_url":"https://api.github.com/repos/timmoon10/lbann/tags","blobs_url":"https://api.github.com/repos/timmoon10/lbann/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/timmoon10/lbann/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/timmoon10/lbann/git/refs{/sha}","trees_url":"https://api.github.com/repos/timmoon10/lbann/git/trees{/sha}","statuses_url":"https://api.github.com/repos/timmoon10/lbann/statuses/{sha}","languages_url":"https://api.github.com/repos/timmoon10/lbann/languages","stargazers_url":"https://api.github.com/repos/timmoon10/lbann/stargazers","contributors_url":"https://api.github.com/repos/timmoon10/lbann/contributors","subscribers_url":"https://api.github.com/repos/timmoon10/lbann/subscribers","subscription_url":"https://api.github.com/repos/timmoon10/lbann/subscription","commits_url":"https://api.github.com/repos/timmoon10/lbann/commits{/sha}","git_commits_url":"https://api.github.com/repos/timmoon10/lbann/git/commits{/sha}","comments_url":"https://api.github.com/repos/timmoon10/lbann/comments{/number}","issue_comment_url":"https://api.github.com/repos/timmoon10/lbann/issues/comments{/number}","contents_url":"https://api.github.com/repos/timmoon10/lbann/contents/{+path}","compare_url":"https://api.github.com/repos/timmoon10/lbann/compare/{base}...{head}","merges_url":"https://api.github.com/repos/timmoon10/lbann/merges","archive_url":"https://api.github.com/repos/timmoon10/lbann/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/timmoon10/lbann/downloads","issues_url":"https://api.github.com/repos/timmoon10/lbann/issues{/number}","pulls_url":"https://api.github.com/repos/timmoon10/lbann/pulls{/number}","milestones_url":"https://api.github.com/repos/timmoon10/lbann/milestones{/number}","notifications_url":"https://api.github.com/repos/timmoon10/lbann/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/timmoon10/lbann/labels{/name}","releases_url":"https://api.github.com/repos/timmoon10/lbann/releases{/id}","deployments_url":"https://api.github.com/repos/timmoon10/lbann/deployments","created_at":"2017-10-19T18:34:41Z","updated_at":"2018-06-15T21:12:10Z","pushed_at":"2018-06-18T23:53:32Z","git_url":"git://github.com/timmoon10/lbann.git","ssh_url":"git@github.com:timmoon10/lbann.git","clone_url":"https://github.com/timmoon10/lbann.git","svn_url":"https://github.com/timmoon10/lbann","homepage":"http://software.llnl.gov/lbann/","size":2408338,"stargazers_count":0,"watchers_count":0,"language":"C++","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":0,"license":{"key":"other","name":"Other","spdx_id":null,"url":null,"node_id":"MDc6TGljZW5zZTA="},"forks":0,"open_issues":0,"watchers":0,"default_branch":"develop"}},"base":{"label":"LLNL:develop","ref":"develop","sha":"acc5375c9a73911331600fb3805e07b4f2200e24","user":{"login":"LLNL","id":5921419,"node_id":"MDEyOk9yZ2FuaXphdGlvbjU5MjE0MTk=","avatar_url":"https://avatars0.githubusercontent.com/u/5921419?v=4","gravatar_id":"","url":"https://api.github.com/users/LLNL","html_url":"https://github.com/LLNL","followers_url":"https://api.github.com/users/LLNL/followers","following_url":"https://api.github.com/users/LLNL/following{/other_user}","gists_url":"https://api.github.com/users/LLNL/gists{/gist_id}","starred_url":"https://api.github.com/users/LLNL/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/LLNL/subscriptions","organizations_url":"https://api.github.com/users/LLNL/orgs","repos_url":"https://api.github.com/users/LLNL/repos","events_url":"https://api.github.com/users/LLNL/events{/privacy}","received_events_url":"https://api.github.com/users/LLNL/received_events","type":"Organization","site_admin":false},"repo":{"id":58576874,"node_id":"MDEwOlJlcG9zaXRvcnk1ODU3Njg3NA==","name":"lbann","full_name":"LLNL/lbann","owner":{"login":"LLNL","id":5921419,"node_id":"MDEyOk9yZ2FuaXphdGlvbjU5MjE0MTk=","avatar_url":"https://avatars0.githubusercontent.com/u/5921419?v=4","gravatar_id":"","url":"https://api.github.com/users/LLNL","html_url":"https://github.com/LLNL","followers_url":"https://api.github.com/users/LLNL/followers","following_url":"https://api.github.com/users/LLNL/following{/other_user}","gists_url":"https://api.github.com/users/LLNL/gists{/gist_id}","starred_url":"https://api.github.com/users/LLNL/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/LLNL/subscriptions","organizations_url":"https://api.github.com/users/LLNL/orgs","repos_url":"https://api.github.com/users/LLNL/repos","events_url":"https://api.github.com/users/LLNL/events{/privacy}","received_events_url":"https://api.github.com/users/LLNL/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/LLNL/lbann","description":"Livermore Big Artificial Neural Network Toolkit","fork":false,"url":"https://api.github.com/repos/LLNL/lbann","forks_url":"https://api.github.com/repos/LLNL/lbann/forks","keys_url":"https://api.github.com/repos/LLNL/lbann/keys{/key_id}","collaborators_url":"https://api.github.com/repos/LLNL/lbann/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/LLNL/lbann/teams","hooks_url":"https://api.github.com/repos/LLNL/lbann/hooks","issue_events_url":"https://api.github.com/repos/LLNL/lbann/issues/events{/number}","events_url":"https://api.github.com/repos/LLNL/lbann/events","assignees_url":"https://api.github.com/repos/LLNL/lbann/assignees{/user}","branches_url":"https://api.github.com/repos/LLNL/lbann/branches{/branch}","tags_url":"https://api.github.com/repos/LLNL/lbann/tags","blobs_url":"https://api.github.com/repos/LLNL/lbann/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/LLNL/lbann/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/LLNL/lbann/git/refs{/sha}","trees_url":"https://api.github.com/repos/LLNL/lbann/git/trees{/sha}","statuses_url":"https://api.github.com/repos/LLNL/lbann/statuses/{sha}","languages_url":"https://api.github.com/repos/LLNL/lbann/languages","stargazers_url":"https://api.github.com/repos/LLNL/lbann/stargazers","contributors_url":"https://api.github.com/repos/LLNL/lbann/contributors","subscribers_url":"https://api.github.com/repos/LLNL/lbann/subscribers","subscription_url":"https://api.github.com/repos/LLNL/lbann/subscription","commits_url":"https://api.github.com/repos/LLNL/lbann/commits{/sha}","git_commits_url":"https://api.github.com/repos/LLNL/lbann/git/commits{/sha}","comments_url":"https://api.github.com/repos/LLNL/lbann/comments{/number}","issue_comment_url":"https://api.github.com/repos/LLNL/lbann/issues/comments{/number}","contents_url":"https://api.github.com/repos/LLNL/lbann/contents/{+path}","compare_url":"https://api.github.com/repos/LLNL/lbann/compare/{base}...{head}","merges_url":"https://api.github.com/repos/LLNL/lbann/merges","archive_url":"https://api.github.com/repos/LLNL/lbann/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/LLNL/lbann/downloads","issues_url":"https://api.github.com/repos/LLNL/lbann/issues{/number}","pulls_url":"https://api.github.com/repos/LLNL/lbann/pulls{/number}","milestones_url":"https://api.github.com/repos/LLNL/lbann/milestones{/number}","notifications_url":"https://api.github.com/repos/LLNL/lbann/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/LLNL/lbann/labels{/name}","releases_url":"https://api.github.com/repos/LLNL/lbann/releases{/id}","deployments_url":"https://api.github.com/repos/LLNL/lbann/deployments","created_at":"2016-05-11T20:04:20Z","updated_at":"2018-06-19T14:52:46Z","pushed_at":"2018-06-19T18:43:29Z","git_url":"git://github.com/LLNL/lbann.git","ssh_url":"git@github.com:LLNL/lbann.git","clone_url":"https://github.com/LLNL/lbann.git","svn_url":"https://github.com/LLNL/lbann","homepage":"http://software.llnl.gov/lbann/","size":11222,"stargazers_count":37,"watchers_count":37,"language":"C++","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":25,"mirror_url":null,"archived":false,"open_issues_count":88,"license":{"key":"other","name":"Other","spdx_id":null,"url":null,"node_id":"MDc6TGljZW5zZTA="},"forks":25,"open_issues":88,"watchers":37,"default_branch":"develop"}},"_links":{"self":{"href":"https://api.github.com/repos/LLNL/lbann/pulls/435"},"html":{"href":"https://github.com/LLNL/lbann/pull/435"},"issue":{"href":"https://api.github.com/repos/LLNL/lbann/issues/435"},"comments":{"href":"https://api.github.com/repos/LLNL/lbann/issues/435/comments"},"review_comments":{"href":"https://api.github.com/repos/LLNL/lbann/pulls/435/comments"},"review_comment":{"href":"https://api.github.com/repos/LLNL/lbann/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/LLNL/lbann/pulls/435/commits"},"statuses":{"href":"https://api.github.com/repos/LLNL/lbann/statuses/50b68b03255fdb1079435084fcbb2569d6e3547c"}},"author_association":"COLLABORATOR"}}
{"actor":{"display_login":"timmoon10"}}