{"action":"created","issue":{"url":"https://api.github.com/repos/dmlc/mxnet/issues/2396","repository_url":"https://api.github.com/repos/dmlc/mxnet","labels_url":"https://api.github.com/repos/dmlc/mxnet/issues/2396/labels{/name}","comments_url":"https://api.github.com/repos/dmlc/mxnet/issues/2396/comments","events_url":"https://api.github.com/repos/dmlc/mxnet/issues/2396/events","html_url":"https://github.com/dmlc/mxnet/issues/2396","id":159803847,"number":2396,"title":"Broadcasting problem?","user":{"login":"magic282","id":2285145,"avatar_url":"https://avatars.githubusercontent.com/u/2285145?v=3","gravatar_id":"","url":"https://api.github.com/users/magic282","html_url":"https://github.com/magic282","followers_url":"https://api.github.com/users/magic282/followers","following_url":"https://api.github.com/users/magic282/following{/other_user}","gists_url":"https://api.github.com/users/magic282/gists{/gist_id}","starred_url":"https://api.github.com/users/magic282/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/magic282/subscriptions","organizations_url":"https://api.github.com/users/magic282/orgs","repos_url":"https://api.github.com/users/magic282/repos","events_url":"https://api.github.com/users/magic282/events{/privacy}","received_events_url":"https://api.github.com/users/magic282/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2016-06-12T02:34:46Z","updated_at":"2016-06-12T06:38:36Z","closed_at":null,"body":"Hi all,\r\n\r\nI am implementing the seq2seq model. I have already implemented a simple one with fixed input/output sequence length. I am trying to use both bucketing and masking since in some tasks like neural machine translation the length of sequence differs a lot.\r\n\r\nFor example in the LSTM encoder:\r\n\r\n```python\r\n# embeding layer\r\ndata = mx.sym.Variable('source')\r\n# label = mx.sym.Variable('softmax_label')\r\nembed = mx.sym.Embedding(data=data, input_dim=self.vocab_size + 1,\r\n                         weight=embed_weight, output_dim=self.embed_dim, name='source_embed')\r\nwordvec = mx.sym.SliceChannel(data=embed, num_outputs=self.seq_len, squeeze_axis=1)\r\nif self.use_masking:\r\n    input_mask = mx.sym.Variable('source_mask')\r\n    masks = mx.sym.SliceChannel(data=input_mask, num_outputs=self.seq_len)\r\n\r\nhidden_all = []\r\nfor seqidx in range(self.seq_len):\r\n    hidden = wordvec[seqidx]\r\n    if self.use_masking:\r\n        mask = masks[seqidx]\r\n\r\n    # stack LSTM\r\n    for i in range(self.num_of_layer):\r\n        if i == 0:\r\n            dp_ratio = 0.\r\n        else:\r\n            dp_ratio = self.dropout\r\n        next_state = lstm(self.state_dim, indata=hidden,\r\n                          prev_state=last_states[i],\r\n                          param=param_cells[i],\r\n                          seqidx=seqidx, layeridx=i, dropout=dp_ratio)\r\n        if self.use_masking:\r\n            # hidden = mx.sym.broadcast_mul((1.0 - mask), hidden) + mx.sym.broadcast_mul(mask, next_state.h)\r\n            hidden = mx.sym.broadcast_mul(mx.sym.broadcast_minus(1.0, mask), hidden) + mx.sym.broadcast_mul(mask, next_state.h)\r\n            # hidden = (1.0 - mask) * hidden + mask * next_state.h\r\n        else:\r\n            hidden = next_state.h\r\n        last_states[i] = next_state\r\n    # decoder\r\n    if self.dropout > 0.:\r\n        hidden = mx.sym.Dropout(data=hidden, p=self.dropout)\r\n    hidden_all.append(hidden)\r\n\r\nreturn hidden_all\r\n```\r\n\r\nWhen I use `hidden = (1.0 - mask) * hidden + mask * next_state.h`\r\nI got the error `mxnet.base.MXNetError: InferShape Error in _mul13's rhs argument` so I assume the operator is not correct here.\r\n\r\nThen I changed it to `hidden = mx.sym.broadcast_mul(mx.sym.broadcast_minus(1.0, mask), hidden) + mx.sym.broadcast_mul(mask, next_state.h)`\r\nI got the error `TypeError: Compose expect Symbol as arguments` so I noticed that the `broadcast_mul` needs two symbols.\r\n\r\nAfter that I tried doing nothing by `hidden = mx.sym.broadcast_mul(mask, next_state.h)` in both encoder and decoder. This time I got another error: `mxnet.base.MXNetError: [10:30:48] D:\\Projects\\opensource\\mxnet\\src\\symbol\\static_graph.cc:488: Check failed: it != grad_map.end() bad graph: Cannot find node slicechannel1's 0-th output`\r\n\r\nAny hints with these errors? \r\nMany thanks!"},"comment":{"url":"https://api.github.com/repos/dmlc/mxnet/issues/comments/225413092","html_url":"https://github.com/dmlc/mxnet/issues/2396#issuecomment-225413092","issue_url":"https://api.github.com/repos/dmlc/mxnet/issues/2396","id":225413092,"user":{"login":"sxjscience","id":5178350,"avatar_url":"https://avatars.githubusercontent.com/u/5178350?v=3","gravatar_id":"","url":"https://api.github.com/users/sxjscience","html_url":"https://github.com/sxjscience","followers_url":"https://api.github.com/users/sxjscience/followers","following_url":"https://api.github.com/users/sxjscience/following{/other_user}","gists_url":"https://api.github.com/users/sxjscience/gists{/gist_id}","starred_url":"https://api.github.com/users/sxjscience/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sxjscience/subscriptions","organizations_url":"https://api.github.com/users/sxjscience/orgs","repos_url":"https://api.github.com/users/sxjscience/repos","events_url":"https://api.github.com/users/sxjscience/events{/privacy}","received_events_url":"https://api.github.com/users/sxjscience/received_events","type":"User","site_admin":false},"created_at":"2016-06-12T06:38:36Z","updated_at":"2016-06-12T06:38:36Z","body":"@magic282 It seems that `hidden` and `next_state.h` have different shapes."}}
{"actor":{"display_login":"sxjscience"}}