title: Generating Input for LSTM from universal sentence encoder output
body: <p>I am working on a multi-class classification problem using LSTM and embeddings obtained from Universal sentence encoder. </p>
  
  <p>Previously I was using Glove embeddings, and I get the required input shape for LSTM (batch_size, timesteps, input_dim). I am planning to use the Universal sentence encoder found that the output of Universal Sentence Encoder is 2d [batch, feature]. How can I make the required changes. </p>
  
  <p>LSTM + Universal sentence encoder</p>
  
  <pre><code>EMBED_SIZE = 512
  
  module_url = "https://tfhub.dev/google/universal-sentence-encoder-large/3"
  embed = hub.Module(module_url)
  
  def UniversalEmbedding(x):
      return embed(tf.squeeze(tf.cast(x, tf.string)), 
              signature="default", as_dict=True)["default"]
  
  
  seq_input = Input(shape=(MAX_SEQUENCE_LENGTH,),dtype='int32')
  print("seq i",seq_input.shape,seq_input)
  
  embedded_seq = Lambda(UniversalEmbedding,                          
                            output_shape=(EMBED_SIZE,))(seq_input)
  print("EMD SEQ",embedding.shape,type(embedded_seq))
  
  # (timesteps, n_features) (,MAX_SEQUENCE_LENGTH, EMBED_SIZE) (,150,512)
  x_1 = LSTM(units=NUM_LSTM_UNITS,
                              name='blstm_1',
                          dropout=DROP_RATE_LSTM)(embedded_seq)
  print(x_1)
  </code></pre>
  
  <p>This produces following error </p>
  
  <pre><code>seq i (?, 150) Tensor("input_8:0", shape=(?, 150), dtype=int32)
  INFO:tensorflow:Saver not created because there are no variables in the graph to restore
  I0529 07:24:32.504808 140127577749376 saver.py:1483] Saver not created because there are no variables in the graph to restore
  EMD SEQ (?, 512) &lt;class 'tensorflow.python.framework.ops.Tensor'&gt;
  ---------------------------------------------------------------------------
  ValueError                                Traceback (most recent call last)
  &lt;ipython-input-34-ea634319205b&gt; in &lt;module&gt;()
       12 x_1 = LSTM(units=NUM_LSTM_UNITS,
       13                             name='blstm_1',
  ---&gt; 14                         dropout=DROP_RATE_LSTM)(embedded_seq)
       15 print(x_1)
       16 
  
  2 frames
  /usr/local/lib/python3.6/dist-packages/keras/engine/base_layer.py in assert_input_compatibility(self, inputs)
      309                                      self.name + ': expected ndim=' +
      310                                      str(spec.ndim) + ', found ndim=' +
  --&gt; 311                                      str(K.ndim(x)))
      312             if spec.max_ndim is not None:
      313                 ndim = K.ndim(x)
  
  ValueError: Input 0 is incompatible with layer blstm_1: expected ndim=3, found ndim=2
  </code></pre>
  
  <p>LSTM + Glove embeddings</p>
  
  <pre><code>embedding_layer = Embedding(nb_words,
                              EMBED_SIZE, 
                              weights=[embedding_matrix],
                              input_length=MAX_SEQUENCE_LENGTH,
                              trainable=False)
  
  seq_input = Input(shape=(MAX_SEQUENCE_LENGTH,),dtype='int32')
  print("SEQ INP",seq_input,seq_input.shape)
  embedded_seq = embedding_layer(seq_input)
  print("EMD SEQ",embedded_seq.shape)
  
  # Bi-directional LSTM  # (timesteps, n_features)
  x_1 = Bidirectional(LSTM(units=NUM_LSTM_UNITS,
                           name='blstm_1',
                           dropout=DROP_RATE_LSTM,
                           recurrent_dropout=DROP_RATE_LSTM),
                      merge_mode='concat')(embedded_seq)
  x_1 = Dropout(DROP_RATE_DENSE)(x_1)
  x_1 = Dense(NUM_DENSE_UNITS,activation='relu')(x_1)
  x_1 = Dropout(DROP_RATE_DENSE)(x_1)
  </code></pre>
  
  <p>OUTPUT (This works properly with LSTM)</p>
  
  <pre><code>SEQ INP Tensor("input_2:0", shape=(?, 150), dtype=int32) (?, 150)
  EMD SEQ (?, 150, 300)
  </code></pre>