title: How to correctly implement a batch-input LSTM network in PyTorch?
body: <p>This <a href="https://github.com/pytorch/pytorch/releases/tag/v0.1.10" rel="noreferrer">release</a> of PyTorch seems provide the <code>PackedSequence</code> for variable lengths of input for recurrent neural network. However, I found it's a bit hard to use it correctly. </p>
  
  <p>Using <code>pad_packed_sequence</code> to recover an output of a RNN layer which were fed by <code>pack_padded_sequence</code>, we got a <code>T x B x N</code> tensor <code>outputs</code> where <code>T</code> is the max time steps, <code>B</code> is the batch size and <code>N</code> is the hidden size. I found that for short sequences in the batch, the subsequent output will be all zeros.</p>
  
  <p>Here are my questions.</p>
  
  <ol>
  <li>For a single output task where the one would need the last output of all the sequences, simple <code>outputs[-1]</code> will give a wrong result since this tensor contains lots of zeros for short sequences. One will need to construct indices by sequence lengths to fetch the individual last output for all the sequences. Is there more simple way to do that?</li>
  <li>For a multiple output task (e.g. seq2seq), usually one will add a linear layer <code>N x O</code> and reshape the batch outputs <code>T x B x O</code> into <code>TB x O</code> and compute the cross entropy loss with the true targets <code>TB</code> (usually integers in language model). In this situation, do these zeros in batch output matters?</li>
  </ol>