title: LSTM one-step-ahead prediction with Tensorflow
body: <p>I am using Tensorflow's combination of <code>GRUCell</code> + <code>MultiRNNCell</code> + <code>dynamic_rnn</code> to generate a multi-layer LSTM to predict a sequence of elements. </p>
  
  <p>In the few examples I have seen, like character-level language models, once the Training stage is done, the Generation seems to be done by feeding only ONE 'character' (or whatever element) at a time to get the next prediction, and then getting the following 'character' based on the first prediction, etc. </p>
  
  <p>My question is, since Tensorflow's <code>dynamic_rnn</code> unrolls the RNN graph into an arbitrary number of steps of whatever sequence length is fed into it, what is the benefit of feeding only one element at a time, once a prediction is gradually being built out? Doesn't it make more sense to be gradually collecting a longer sequence with each predictive step and re-feeding it into the graph? I.e. after generating the first prediction, feed back a sequence of 2 elements, and then 3, etc.? </p>
  
  <p>I am currently trying out the prediction stage by initially feeding in a sequence of 15 elements (actual historic data), getting the last element of the prediction, and then replacing one element in the original input with that predicted value, and so on in a loop of N predictive steps.</p>
  
  <p>What is the disadvantage of this approach versus feeding just one element at-a-time?</p>