{"action":"created","issue":{"url":"https://api.github.com/repos/BVLC/caffe/issues/3612","repository_url":"https://api.github.com/repos/BVLC/caffe","labels_url":"https://api.github.com/repos/BVLC/caffe/issues/3612/labels{/name}","comments_url":"https://api.github.com/repos/BVLC/caffe/issues/3612/comments","events_url":"https://api.github.com/repos/BVLC/caffe/issues/3612/events","html_url":"https://github.com/BVLC/caffe/pull/3612","id":129852271,"number":3612,"title":"Tied weights with transpose flag for InnerProduct layer","user":{"login":"kashefy","id":6430736,"avatar_url":"https://avatars.githubusercontent.com/u/6430736?v=3","gravatar_id":"","url":"https://api.github.com/users/kashefy","html_url":"https://github.com/kashefy","followers_url":"https://api.github.com/users/kashefy/followers","following_url":"https://api.github.com/users/kashefy/following{/other_user}","gists_url":"https://api.github.com/users/kashefy/gists{/gist_id}","starred_url":"https://api.github.com/users/kashefy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kashefy/subscriptions","organizations_url":"https://api.github.com/users/kashefy/orgs","repos_url":"https://api.github.com/users/kashefy/repos","events_url":"https://api.github.com/users/kashefy/events{/privacy}","received_events_url":"https://api.github.com/users/kashefy/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":10,"created_at":"2016-01-29T18:55:41Z","updated_at":"2016-02-08T10:43:38Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/BVLC/caffe/pulls/3612","html_url":"https://github.com/BVLC/caffe/pull/3612","diff_url":"https://github.com/BVLC/caffe/pull/3612.diff","patch_url":"https://github.com/BVLC/caffe/pull/3612.patch"},"body":"I wanted to train an autoencoder where the deocder uses the tranpose of the encoder's weight matrix. This is first discussed in [#670](https://github.com/BVLC/caffe/issues/670) and followed up in [#1211 (comment)](https://github.com/BVLC/caffe/issues/1211#issuecomment-58031848). But it seemed this wasn't resolved. I found @jeffdonahue 's suggestion in [this comment](https://github.com/BVLC/caffe/issues/1211#issuecomment-58038944) to just add a transpose flag to the InnerProduct layer quite reasonable.\r\n\r\nThis PR adds a transpose flag to the InnerProduct layer as well as its params protobuf message.\r\nWhen set to true for the deocder, in the forward pass, the call to the matrix multiplication routine is instructed to **NOT** transpose the weight matrix. Which is what you want in the usual case and for the encoder.\r\n\r\nTying the weights between encoder and decoder requires:\r\n\r\n1. sharing the encoder's weight params with the inner product layer that is the decoder.\r\n2. setting the share_mode to true in both ip layers (otherwise it won't allow for the shape mismatch)\r\n3. Adding 'transpose: true' to the decoder's inner_product_param\r\n\r\nA sample [trainval.prototxt](https://gist.github.com/kashefy/72205d7c2da39520f893) to demonstrate usage.\r\n\r\nI haven't written unit tests around this yet. Open to suggestions to what makes sense to test for here.\r\n\r\nThanks for reviewing and looking forward to the feedback."},"comment":{"url":"https://api.github.com/repos/BVLC/caffe/issues/comments/181300789","html_url":"https://github.com/BVLC/caffe/pull/3612#issuecomment-181300789","issue_url":"https://api.github.com/repos/BVLC/caffe/issues/3612","id":181300789,"user":{"login":"kashefy","id":6430736,"avatar_url":"https://avatars.githubusercontent.com/u/6430736?v=3","gravatar_id":"","url":"https://api.github.com/users/kashefy","html_url":"https://github.com/kashefy","followers_url":"https://api.github.com/users/kashefy/followers","following_url":"https://api.github.com/users/kashefy/following{/other_user}","gists_url":"https://api.github.com/users/kashefy/gists{/gist_id}","starred_url":"https://api.github.com/users/kashefy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kashefy/subscriptions","organizations_url":"https://api.github.com/users/kashefy/orgs","repos_url":"https://api.github.com/users/kashefy/repos","events_url":"https://api.github.com/users/kashefy/events{/privacy}","received_events_url":"https://api.github.com/users/kashefy/received_events","type":"User","site_admin":false},"created_at":"2016-02-08T10:43:38Z","updated_at":"2016-02-08T10:43:38Z","body":"Travis job passing now, can't really explain why. But glad it the import errors are gone now."}}