title: Keras Bidirectional LSTM - Layer grouping
body: <p>While working to implement a paper (<em>Dialogue Act Sequence Labeling using Hierarchical encoder with CRF</em>) using Keras, I need to implement a specific Bidirectional LSTM architecture.</p>
  
  <p>I have to train the network on the concept of a Conversation. Conversations are composed of Utterances, and Utterances are composed of Words. Words are N-dimensional vectors. The model represented in the paper first reduces each Utterance to a single <code>M</code>-dimensional vector. To achieve this, it uses a Bidirectional LSTM layer. Let's call this layer <strong>A</strong>.</p>
  
  <p>(For simplicity, let's assume that each Utterance has a length of <code>|U|</code> and each Conversation has a length of <code>|C|</code>)</p>
  
  <p>Each Utterance is input to a Bi-LSTM layer with U timesteps, and the output of the last timestep is taken. The input size is <code>(|U|, N)</code>, and the output size is <code>(1, M)</code>.</p>
  
  <p>This Bi-LSTM layer should be applied separately/simultaneously to each Utterance in the Conversation. Note that, since the network takes as input the entire Conversation, the dimensions for a single input to the network would be <code>(|C|, |U|, N)</code>.</p>
  
  <p>As the paper describes, I intend to feed each utterance (i.e. each <code>(|U|, N)</code>) of that input and feed it to a Bi-LSTM layer with <code>|U|</code> units. As there are |C| Utterances in a Conversation, this implies that there should be a total of <code>|C| x |U|</code> Bi-LSTM units, grouped into <code>|C|</code> different partitions for each Utterance. There should be no connection between the <code>|C|</code> groups of units. Once processed, the output of each of those <code>C</code> groups of Bidirectional LSTM units will then be fed into another Bi-LSTM layer, say <strong>B</strong>.</p>
  
  <p>How is it possible to feed specific portions of the input only to specific portions of the layer <strong>A</strong>, and make sure that they are not interconnected? (i.e. the portion of Bi-LSTM units used for an Utterance should not be connected to the Bi-LSTM units used for another Utterance)</p>
  
  <p>Is it possible to achieve this through <code>keras.models.Sequential</code>, or is there a specific way to achieve this using Functional API?</p>
  
  <p>Here is what I have tried so far:</p>
  
  <pre><code># ...
  model = Sequential()
  model.add(Bidirectional(LSTM(C * U), input_shape = (C, U, N),
                          merge_mode='concat'))
  model.add(GlobalMaxPooling1D())
  model.add(Bidirectional(LSTM(n, return_sequences = True), merge_mode='concat'))
  # ...
  
  model.compile(loss = loss_function,
                optimizer = optimizer,
                metrics=['accuracy'])
  </code></pre>
  
  <p>However, this code is currently receiving the following error:</p>
  
  <blockquote>
    <p>ValueError: Input 0 is incompatible with layer bidirectional_1: expected ndim=3, found ndim=4</p>
  </blockquote>
  
  <p>More importantly, the code above obviously does not do the grouping I mentioned. I am looking for a way to enhance the model as I described above.</p>
  
  <p>Finally, below is the figure of the model I described above. It may possibly help clarify some of the written content narrated above. The layer tagged as "Utterance layer" is what I called the layer <strong>A</strong>. As you can see in the figure, each utterance <code>u_i</code> in the figure is composed of words <code>w_j</code>, which are N-dimensional vectors. (You may omit the embedding layer for the purposes of this question) Assuming, for simplicity, that each <code>u_i</code> has equal number of Words, then each <em>group</em> of Bidirectional LSTM nodes in the Utterance Layer will have an input size of <code>(|U|, N)</code>. Yet, since there are <code>|C|</code> such utterances <code>u_i</code> in a Conversation, the dimensions of the entire input will be <code>(|C|, |U|, N)</code>.</p>
  
  <p><a href="https://i.stack.imgur.com/xL1IN.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/xL1IN.jpg" alt="Bi-LSTM Model diagram"></a></p>