{"action":"created","issue":{"url":"https://api.github.com/repos/keras-team/keras/issues/11000","repository_url":"https://api.github.com/repos/keras-team/keras","labels_url":"https://api.github.com/repos/keras-team/keras/issues/11000/labels{/name}","comments_url":"https://api.github.com/repos/keras-team/keras/issues/11000/comments","events_url":"https://api.github.com/repos/keras-team/keras/issues/11000/events","html_url":"https://github.com/keras-team/keras/issues/11000","id":354143916,"node_id":"MDU6SXNzdWUzNTQxNDM5MTY=","number":11000,"title":"Keras Error when input Dense layer with a reshaped layer","user":{"login":"dzhao123","id":39663377,"node_id":"MDQ6VXNlcjM5NjYzMzc3","avatar_url":"https://avatars2.githubusercontent.com/u/39663377?v=4","gravatar_id":"","url":"https://api.github.com/users/dzhao123","html_url":"https://github.com/dzhao123","followers_url":"https://api.github.com/users/dzhao123/followers","following_url":"https://api.github.com/users/dzhao123/following{/other_user}","gists_url":"https://api.github.com/users/dzhao123/gists{/gist_id}","starred_url":"https://api.github.com/users/dzhao123/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dzhao123/subscriptions","organizations_url":"https://api.github.com/users/dzhao123/orgs","repos_url":"https://api.github.com/users/dzhao123/repos","events_url":"https://api.github.com/users/dzhao123/events{/privacy}","received_events_url":"https://api.github.com/users/dzhao123/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-08-27T00:34:49Z","updated_at":"2018-08-30T07:40:48Z","closed_at":"2018-08-30T07:40:48Z","author_association":"NONE","body":"I am trying to implement a lstm model which process 100 texts and then concatenate them together to feed into a dense layer. However there comes an error, which is:\r\n\r\n`TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'`\r\n\r\nThe code shows below:\r\n\r\n```\r\nfrom keras.layers import Embedding, Dense, LSTM, Masking, concatenate, Flatten, Activation, RepeatVector, Permute, Multiply, Lambda, TimeDistributed, Reshape\r\nfrom keras import Input, Model\r\nfrom keras import backend as K\r\n\r\n\r\nlstm = LSTM(100)\r\nembd = Embedding(1000, 128)\r\ndense = Dense(1, activation='tanh')#(lstm_outputs) \r\n\r\n\r\ninputs = []\r\noutputs = []\r\n\r\n\r\nfor _ in range(2):\r\n    encoder_inputs = Input(shape=(160,))\r\n    print(encoder_inputs.shape)\r\n    masked_outputs = Masking(0)(encoder_inputs)\r\n    encoder_embd = embd(masked_outputs)\r\n    print(encoder_embd.shape)\r\n    lstm_outputs = lstm(encoder_embd)\r\n    attention = dense(lstm_outputs)\r\n    attention = Activation('softmax')(attention)\r\n    attention = RepeatVector(100)(attention)\r\n    attention = Permute([2, 1])(attention)\r\n\r\n    utterance_representation = Multiply()([lstm_outputs, attention])\r\n    print(utterance_representation.shape)\r\n\r\n    utterance_representation = K.squeeze(utterance_representation,axis=1)\r\n\r\n    utterance_outputs = K.reshape(utterance_representation, [-1, K.shape(utterance_representation)[0]*K.shape(utterance_representation)[1]])\r\n\r\n    encoder_outputs = Dense(200, activation='relu')(utterance_outputs)\r\n    model_outputs = Dense(1, activation='softmax')(encoder_outputs)\r\n\r\n    inputs.append(encoder_inputs)\r\n    outputs.append(model_outputs)\r\n\r\n\r\nencoder_outputs = Dense(200, activation='relu')(outputs_one)\r\nmodel_outputs = Dense(1, activation='softmax')(encoder_outputs)\r\n\r\nmodel = Model(input=inputs, output=outputs)\r\nmodel.compile(optimizer='rmsprop', loss='mse', metrics=['accuracy'])\r\n\r\n\r\nmodel.fit_generator(feed_generator('data.h5'), steps_per_epoch=250, epochs=1, verbose=1, callbacks=[EarlyStopping(monitor='val_loss', patience=4)])\r\n```\r\nCan anyone gives some help, thanks!\r\n"},"comment":{"url":"https://api.github.com/repos/keras-team/keras/issues/comments/417221468","html_url":"https://github.com/keras-team/keras/issues/11000#issuecomment-417221468","issue_url":"https://api.github.com/repos/keras-team/keras/issues/11000","id":417221468,"node_id":"MDEyOklzc3VlQ29tbWVudDQxNzIyMTQ2OA==","user":{"login":"maxpumperla","id":3462566,"node_id":"MDQ6VXNlcjM0NjI1NjY=","avatar_url":"https://avatars1.githubusercontent.com/u/3462566?v=4","gravatar_id":"","url":"https://api.github.com/users/maxpumperla","html_url":"https://github.com/maxpumperla","followers_url":"https://api.github.com/users/maxpumperla/followers","following_url":"https://api.github.com/users/maxpumperla/following{/other_user}","gists_url":"https://api.github.com/users/maxpumperla/gists{/gist_id}","starred_url":"https://api.github.com/users/maxpumperla/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/maxpumperla/subscriptions","organizations_url":"https://api.github.com/users/maxpumperla/orgs","repos_url":"https://api.github.com/users/maxpumperla/repos","events_url":"https://api.github.com/users/maxpumperla/events{/privacy}","received_events_url":"https://api.github.com/users/maxpumperla/received_events","type":"User","site_admin":false},"created_at":"2018-08-30T07:40:48Z","updated_at":"2018-08-30T07:40:48Z","author_association":"COLLABORATOR","body":"@dzhao123 after you squeeze `utterance_representation` has shape `(?, 100)`, which makes `K.shape(utterance_representation)[0]*K.shape(utterance_representation)[1]` evaluate to None, which in turn makes `fan_in` of your Dense layer `None`, which breaks weight init. The error that ultimately causes this is\r\n\r\n```python\r\nscale /= max(1., float(fan_in + fan_out) / 2)\r\n```\r\n\r\nsince you can't add ints and None.\r\n\r\nThe flexibility `Reshape` gives you comes at a cost: you need to carefully evaluate if the shapes you provide make sense. Replace your line above with \r\n\r\n```python\r\n    utterance_outputs = K.reshape(utterance_representation, [-1, 100])\r\n```\r\n\r\nand watch it succeed. Note that I have no clue what `outputs_one` is though."}}
{"actor":{"display_login":"maxpumperla"}}