<p>You will need a variable number of outputs which would require a recurrent network on the prediction side. Let's try building one on the existing network:</p>

<pre><code># first we'll add an extra input telling how many outputs we need
num_outs = Input(shape=(1,), dtype='int')
# ... continuing from
answer = LSTM(32)(answer)  # (samples, 32)
# answer is your encoded context-query, we will decode it into a sequence
answers = RepeatVector(num_outs[0])(answer) # (samples, num_outs, 32)
# Another RNN to give us decoding (optional)
answers = LSTM(32, return_sequences=True)(answers) # note return_sequences
answers = TimeDistributed(Dense(vocab_size, activation='softmax'))(answers)
# we have (samples, num_outs, vocab_size) so num_outs words
# ...
</code></pre>

<p>Now your targets have to be 3D shape as well. <em>Important</em>, you have to append a <strong>end-of-answer</strong> token to every answer so you know when to stop at prediction time. Equally, you would pad the number of answers within a batch to get a tensor after end-of-answer token.</p>

<p>Now at prediction time, you can ask for 10 words and chop words after end-of-answer token, similar to how machine translation is done using seq2seq models. For reference, have a look at <a href="https://arxiv.org/abs/1506.07285" rel="nofollow noreferrer">Dynamic Memory Networks</a>.</p>
Error when checking target: expected activation_29 to have shape (1,) but got array with shape (3,)
<p>I am trying to modify <a href="https://github.com/keras-team/keras/blob/master/examples/babi_memnn.py" rel="nofollow noreferrer">Keras's memory neural net using the bAbI dataset</a> from outputting a single word to outputting multiple words (3 in this example). For context, this is an NLP model using LSTM for question answering. </p>

<p>Here is a snippet of the model structure:</p>

<pre><code># placeholders
input_sequence = Input((story_maxlen,))
question = Input((query_maxlen,))

# encoders
# embed the input sequence into a sequence of vectors
input_encoder_m = Sequential()
input_encoder_m.add(Embedding(input_dim=vocab_size,
                              output_dim=64))
input_encoder_m.add(Dropout(0.3))
# output: (samples, story_maxlen, embedding_dim)

# embed the input into a sequence of vectors of size query_maxlen
input_encoder_c = Sequential()
input_encoder_c.add(Embedding(input_dim=vocab_size,
                              output_dim=query_maxlen))
input_encoder_c.add(Dropout(0.3))
# output: (samples, story_maxlen, query_maxlen)

# embed the question into a sequence of vectors
question_encoder = Sequential()
question_encoder.add(Embedding(input_dim=vocab_size,
                               output_dim=64,
                               input_length=query_maxlen))
question_encoder.add(Dropout(0.3))
# output: (samples, query_maxlen, embedding_dim)

# encode input sequence and questions (which are indices)
# to sequences of dense vectors
input_encoded_m = input_encoder_m(input_sequence)
input_encoded_c = input_encoder_c(input_sequence)
question_encoded = question_encoder(question)

# compute a 'match' between the first input vector sequence
# and the question vector sequence
# shape: `(samples, story_maxlen, query_maxlen)`
match = dot([input_encoded_m, question_encoded], axes=(2, 2))
match = Activation('softmax')(match)

# add the match matrix with the second input vector sequence
response = add([match, input_encoded_c])  # (samples, story_maxlen, query_maxlen)
response = Permute((2, 1))(response)  # (samples, query_maxlen, story_maxlen)

# concatenate the match matrix with the question vector sequence
answer = concatenate([response, question_encoded])

# the original paper uses a matrix multiplication for this reduction step.
# we choose to use a RNN instead.
answer = LSTM(32)(answer)  # (samples, 32)

# one regularization layer -- more would probably be needed.
answer = Dropout(0.3)(answer)
answer = Dense(vocab_size)(answer)  # (samples, vocab_size)
# we output a probability distribution over the vocabulary
answer = Activation('softmax')(answer)
</code></pre>

<p>and this is how it is being compiled and trained:</p>

<pre><code>model = Model([input_sequence, question], answer)
model.compile(optimizer='rmsprop', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
model.fit([inputs_train, queries_train], answers_train,
          batch_size=32,
          epochs=num_epochs,
          validation_data=([inputs_test, queries_test], answers_test))
</code></pre>

<p>In the above example, the answers_train variable is a 1xn matrix where each item is the value for a question. So, for example, the first three answers:</p>

<pre><code>print(answers_train[:3])
</code></pre>

<p>outputs:</p>

<pre><code>[16 16 19]
</code></pre>

<h2>My Issue</h2>

<p>This is the change I made was to the answer_train variable where:</p>

<pre><code>print(answers_train[:3])
</code></pre>

<p>outputs:</p>

<pre><code>[[ 0  0 16]
 [ 0  0 27]
 [ 0  0 16]]
</code></pre>

<p>basically, I'm trying to get up to three words predicted instead of one.</p>

<p>When I do this and try to train the model I get this error:</p>

<blockquote>
  <p>ValueError: Error when checking target: expected activation_29 to have
  shape (1,) but got array with shape (3,)</p>
</blockquote>

<p>Here is the output of model.summary():</p>

<pre><code>__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_1 (InputLayer)            (None, 552)          0                                            
__________________________________________________________________________________________________
input_2 (InputLayer)            (None, 5)            0                                            
__________________________________________________________________________________________________
sequential_1 (Sequential)       multiple             2304        input_1[0][0]                    
__________________________________________________________________________________________________
sequential_3 (Sequential)       (None, 5, 64)        2304        input_2[0][0]                    
__________________________________________________________________________________________________
dot_1 (Dot)                     (None, 552, 5)       0           sequential_1[1][0]               
                                                                 sequential_3[1][0]               
__________________________________________________________________________________________________
activation_1 (Activation)       (None, 552, 5)       0           dot_1[0][0]                      
__________________________________________________________________________________________________
sequential_2 (Sequential)       multiple             180         input_1[0][0]                    
__________________________________________________________________________________________________
add_1 (Add)                     (None, 552, 5)       0           activation_1[0][0]               
                                                                 sequential_2[1][0]               
__________________________________________________________________________________________________
permute_1 (Permute)             (None, 5, 552)       0           add_1[0][0]                      
__________________________________________________________________________________________________
concatenate_1 (Concatenate)     (None, 5, 616)       0           permute_1[0][0]                  
                                                                 sequential_3[1][0]               
__________________________________________________________________________________________________
lstm_1 (LSTM)                   (None, 32)           83072       concatenate_1[0][0]              
__________________________________________________________________________________________________
dropout_4 (Dropout)             (None, 32)           0           lstm_1[0][0]                     
__________________________________________________________________________________________________
dense_1 (Dense)                 (None, 36)           1188        dropout_4[0][0]                  
__________________________________________________________________________________________________
activation_2 (Activation)       (None, 36)           0           dense_1[0][0]                    
==================================================================================================
Total params: 89,048
Trainable params: 89,048
Non-trainable params: 0
__________________________________________________________________________________________________
</code></pre>

<p>What I understand is that the model was built to determine a single word answer (i.e. shape (1,)) and that I need to modify the model because now I expect it to determine multiple word answers (in this case, shape (3,)). What I don't understand is how to change the model structure to accomplish that.</p>

<p>I don't see anywhere in the model's summary that indicates where the shape (1,) is defined. I only see definitions for the max story size in words (552), the max query/question size in words (5), and the vocabulary size in words (36).</p>

<p>Is anyone able to help me figure out what I'm doing wrong?</p>

<hr>

<h2>Update #1</h2>

<p>I've learned a few more things while I've been continuing to research this problem. I could be wrong on all these points as I'm not familiar with the fine details of ML and NNs so feel free to call me out on anything that seems amiss.</p>

<ul>
<li>The last dense layer of shape (None, 36) is sized based on the vocabulary size and the subsequent softmax activation layer's purpose is to produce a vector of probabilities to indicate which word is the correct one. If that's the case then by reducing the last dense layer to (None, 3) am I losing information? Would I be just getting a vector of three probabilities without any indication as to what words they apply to? Unless the last dense layer are the indicies of the vectorized vocabulary? In that case I'd know the words being predicted, but then what would be the purpose of the subsequent activation layer?</li>
<li>The <code>sparse_categorical_crossentropy</code> loss function reduces the shape of the final output to (1,) in <a href="https://github.com/keras-team/keras/blob/7365a99f6e832847808c7aa28718d32fbc744b21/keras/engine/training.py#L770" rel="nofollow noreferrer">~/keras/engine/training.py on line 770</a>. Does that mean I'm using the wrong loss function? I can't use <code>categorical_crossentropy</code> because I don't want to have a one-hot vectored output. Does that mean I need to change the whole model altogether or will another loss function give me the desired output?</li>
</ul>

<p>I guess in summation, is a tweak to the model even possible or do I need to use a completely different model? If you could also provide clarity on my confusion based on the above two points I would be very grateful.</p>