title: How can we define one-to-one, one-to-many, many-to-one, and many-to-many LSTM neural networks in Keras?
body: <p>I am reading <a href="http://karpathy.github.io/2015/05/21/rnn-effectiveness/" rel="noreferrer">this</a> article (The Unreasonable Effectiveness of Recurrent Neural Networks) and want to understand how to express one-to-one, one-to-many, many-to-one, and many-to-many LSTM neural networks in Keras. I have read a lot about RNN and understand how LSTM NNs work, in particular vanishing gradient, LSTM cells, their outputs and states, sequence output and etc. However, I have trouble expressing all these concepts in Keras.</p>
  
  <p>To start with I have created the following toy NN using LSTM layer</p>
  
  <pre><code>from keras.models import Model
  from keras.layers import Input, LSTM
  import numpy as np
  
  t1 = Input(shape=(2, 3))
  t2 = LSTM(1)(t1)
  model = Model(inputs=t1, outputs=t2)
  
  inp = np.array([[[1,2,3],[4,5,6]]])
  model.predict(inp)
  </code></pre>
  
  <p>Output:</p>
  
  <pre><code>array([[ 0.0264638]], dtype=float32)
  </code></pre>
  
  <p>In my example I have the input shape 2 by 3. As far as I understand this means that the input is a sequence of 2 vectors and each vector has 3 features and hence my input must be a 3D tensor of shape <code>(n_examples, 2, 3)</code>. In terms of 'sequences', the input is a sequence of length 2, and each element in the sequence is expressed by 3 features (please correct me if I am wrong). When I call <code>predict</code> it returns a 2-dim tensor with a single scalar. So,</p>
  
  <p>Q1: Is it one-to-one or another type of LSTM network?  </p>
  
  <p>When we say "one/many input and one/many output" </p>
  
  <p>Q2: what do we mean by "one/many input/output"? A "one/many" scalar(s), vector(s), sequence(s)..., one/many what?      </p>
  
  <p>Q3: Can someone give a simple working example in Keras for each type of the networks: 1-1, 1-M, M-1, and M-M?</p>
  
  <p>PS: I ask multiple questions in a single thread since they are very close and related to each other.</p>