{"action":"created","issue":{"url":"https://api.github.com/repos/fchollet/keras/issues/1629","repository_url":"https://api.github.com/repos/fchollet/keras","labels_url":"https://api.github.com/repos/fchollet/keras/issues/1629/labels{/name}","comments_url":"https://api.github.com/repos/fchollet/keras/issues/1629/comments","events_url":"https://api.github.com/repos/fchollet/keras/issues/1629/events","html_url":"https://github.com/fchollet/keras/issues/1629","id":130920058,"number":1629,"title":"How to implement a deep bidirectional LSTM?","user":{"login":"udani969","id":13414903,"avatar_url":"https://avatars.githubusercontent.com/u/13414903?v=3","gravatar_id":"","url":"https://api.github.com/users/udani969","html_url":"https://github.com/udani969","followers_url":"https://api.github.com/users/udani969/followers","following_url":"https://api.github.com/users/udani969/following{/other_user}","gists_url":"https://api.github.com/users/udani969/gists{/gist_id}","starred_url":"https://api.github.com/users/udani969/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/udani969/subscriptions","organizations_url":"https://api.github.com/users/udani969/orgs","repos_url":"https://api.github.com/users/udani969/repos","events_url":"https://api.github.com/users/udani969/events{/privacy}","received_events_url":"https://api.github.com/users/udani969/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":12,"created_at":"2016-02-03T05:47:05Z","updated_at":"2016-09-09T08:08:03Z","closed_at":"2016-02-03T14:51:32Z","body":"I am trying to implement a LSTM based speech recognizer. So far I could set up bidirectional LSTM (i think it is working as a bidirectional LSTM) by following the example in Merge layer. Now I want to try it with another bidirectional LSTM layer, which make it a deep bidirectional LSTM. But I am unable to figure out how to connect the output of the previously merged two layers into a second set of LSTM layers. I don't know whether it is possible with Keras. Hope someone can help me with this.\r\n\r\nCode for my single layer bidirectional LSTM is as follows\r\n```\r\nleft = Sequential()\r\nleft.add(LSTM(output_dim=hidden_units, init='uniform', inner_init='uniform',\r\n               forget_bias_init='one', return_sequences=True, activation='tanh',\r\n               inner_activation='sigmoid', input_shape=(99, 13)))\r\nright = Sequential()\r\nright.add(LSTM(output_dim=hidden_units, init='uniform', inner_init='uniform',\r\n               forget_bias_init='one', return_sequences=True, activation='tanh',\r\n               inner_activation='sigmoid', input_shape=(99, 13), go_backwards=True))\r\n\r\nmodel = Sequential()\r\nmodel.add(Merge([left, right], mode='sum'))\r\n\r\nmodel.add(TimeDistributedDense(nb_classes))\r\nmodel.add(Activation('softmax'))\r\n\r\nsgd = SGD(lr=0.1, decay=1e-5, momentum=0.9, nesterov=True)\r\nmodel.compile(loss='categorical_crossentropy', optimizer=sgd)\r\nprint(\"Train...\")\r\nmodel.fit([X_train, X_train], Y_train, batch_size=1, nb_epoch=nb_epoches, validation_data=([X_test, X_test], Y_test), verbose=1, show_accuracy=True)\r\n```\r\n\r\nDimensions of my x and y values are as follows. \r\n\r\n(100, 'train sequences')\r\n(20, 'test sequences')\r\n('X_train shape:', (100, 99, 13))\r\n('X_test shape:', (20, 99, 13))\r\n('y_train shape:', (100, 99, 11))\r\n('y_test shape:', (20, 99, 11))"},"comment":{"url":"https://api.github.com/repos/fchollet/keras/issues/comments/245847867","html_url":"https://github.com/fchollet/keras/issues/1629#issuecomment-245847867","issue_url":"https://api.github.com/repos/fchollet/keras/issues/1629","id":245847867,"user":{"login":"strin","id":2183232,"avatar_url":"https://avatars.githubusercontent.com/u/2183232?v=3","gravatar_id":"","url":"https://api.github.com/users/strin","html_url":"https://github.com/strin","followers_url":"https://api.github.com/users/strin/followers","following_url":"https://api.github.com/users/strin/following{/other_user}","gists_url":"https://api.github.com/users/strin/gists{/gist_id}","starred_url":"https://api.github.com/users/strin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/strin/subscriptions","organizations_url":"https://api.github.com/users/strin/orgs","repos_url":"https://api.github.com/users/strin/repos","events_url":"https://api.github.com/users/strin/events{/privacy}","received_events_url":"https://api.github.com/users/strin/received_events","type":"User","site_admin":false},"created_at":"2016-09-09T08:08:03Z","updated_at":"2016-09-09T08:08:03Z","body":"@farizrahman4u @9thDimension when running lstm in the reverse direction, shouldn't the output corresponds to input_n, input_{n-1}, input_{n-2}, ..., input_1? In that case, when concatenating with the output from the forward direction, we should reverse it?"}}
{"actor":{"display_login":"strin"}}