title: LSTM state within a batch
body: <p>I'm currently working on a <a href="http://machinelearningmastery.com/understanding-stateful-lstm-recurrent-neural-networks-python-keras/" rel="nofollow noreferrer">Keras tutorial for recurrent network training</a> and I'm having trouble understanding the Stateful LSTM concept. To keep things as simple as possible, the sequences have the same length <code>seq_length</code>. As far as I get it, the input data is of shape <code>(n_samples, seq_length, n_features)</code> and we then train our LSTM on <code>n_samples/M</code> batches of size <code>M</code> as follows:</p>
  
  <p>For each batch:</p>
  
  <ol>
  <li>Feed in the 2D-tensors <code>(seq_length, n_features)</code> and for each input 2D-tensor compute the gradient</li>
  <li>Sum these gradients to get the total gradient on the batch</li>
  <li>Backpropagate the gradient and update weights</li>
  </ol>
  
  <p>In the tutorial's example, feeding in the 2D-tensors is feeding in a sequence of size <code>seq_length</code> of letters encoded as vectors of length <code>n_features</code>. However, the tutorial says that in the Keras implementation of LSTMs, the hidden state is not reset after a whole sequence (2D-tensor) is fed in, but after a batch of sequences is fed in to use more context. </p>
  
  <p>Why does keeping the hidden state of the previous sequence and using it as initial hidden state for our current sequence improve the learning and the predictions on our test set, since that "previously learned" initial hidden state won't be available when making predictions ? Moreover, Keras' default behaviour is to shuffle input samples at the beginning of each epoch so the batch context is changed at each epoch. This behaviour seems contradictory to keeping the hidden state through a batch since batch context is random.  </p>