title: RNN: Back-propagation through time when output is taken only at final timestep
body: <p>In this <a href="http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-1-introduction-to-rnns/" rel="nofollow noreferrer">blog</a> on Recurrent Neural Networks by Denny Britz.
  <a href="https://i.stack.imgur.com/R5nRD.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/R5nRD.jpg" alt="enter image description here"></a>
   Author states that, "<em>The above diagram has outputs at each time step, but depending on the task this may not be necessary. For example, when predicting the sentiment of a sentence we may only care about the final output, not the sentiment after each word. Similarly, we may not need inputs at each time step.</em>"</p>
  
  <p>In the case when we take output only at the final timestep: How will backpropogation change, if there are no outputs at each time step, only the final one? We need to define loss at each time step, but how to do it without outputs?</p>