{"action":"opened","issue":{"url":"https://api.github.com/repos/fchollet/keras/issues/4219","repository_url":"https://api.github.com/repos/fchollet/keras","labels_url":"https://api.github.com/repos/fchollet/keras/issues/4219/labels{/name}","comments_url":"https://api.github.com/repos/fchollet/keras/issues/4219/comments","events_url":"https://api.github.com/repos/fchollet/keras/issues/4219/events","html_url":"https://github.com/fchollet/keras/issues/4219","id":185926316,"number":4219,"title":"Sample weights for hierarchical seq2seq","user":{"login":"ivodopyanov","id":1856493,"avatar_url":"https://avatars.githubusercontent.com/u/1856493?v=3","gravatar_id":"","url":"https://api.github.com/users/ivodopyanov","html_url":"https://github.com/ivodopyanov","followers_url":"https://api.github.com/users/ivodopyanov/followers","following_url":"https://api.github.com/users/ivodopyanov/following{/other_user}","gists_url":"https://api.github.com/users/ivodopyanov/gists{/gist_id}","starred_url":"https://api.github.com/users/ivodopyanov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ivodopyanov/subscriptions","organizations_url":"https://api.github.com/users/ivodopyanov/orgs","repos_url":"https://api.github.com/users/ivodopyanov/repos","events_url":"https://api.github.com/users/ivodopyanov/events{/privacy}","received_events_url":"https://api.github.com/users/ivodopyanov/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2016-10-28T13:42:59Z","updated_at":"2016-10-28T13:42:59Z","closed_at":null,"body":"Hello. I intend to use system of two hierarchical seq2seq models for unsupervised training. It should look something like that:\r\n\r\nmodel = Sequential()\r\nmodel.add(Layer(batch_input_shape=(PARAGRAPH_LEN, SENTENCE_LEN, WORD_LEN, CHAR_COUNT)))\r\nmodel.add(TimeDistributed(LSTM(HIDDEN_LAYER_1_SIZE)))\r\nmodel.add(LSTM(HIDDEN_LAYER_2_SIZE))\r\nmodel.add(RepeatVector(SENTENCE_LEN))\r\nmodel.add(LSTM(HIDDEN_LAYER_2_SIZE, return_sequences=True))\r\nmodel.add(TimeDistributed(Dense(HIDDEN_LAYER_1_SIZE)))\r\nmodel.add(TimeDistributed(RepeatVector(WORD_LEN)))\r\nmodel.add(TimeDistributed(LSTM(HIDDEN_LAYER_1_SIZE, return_sequences=True)))\r\nmodel.add(TimeDistributed(TimeDistributed(Dense(CHAR_COUNT, activation='softmax'))))\r\n\r\nWhat I'm trying to achieve is to get char-based word embedding from 1st LSTM layer and then feed them into 2nd LSTM layer to generate sentence embedding. For that I use seq2seq as autoencoders (so, X == Y).\r\n\r\nCan I use sample weights for such data to mask out paddings in Y? Wiki says it supports only 1D and 2D sample weights but here they should be 3D."}}
{"actor":{"display_login":"ivodopyanov"}}