title: Understanding a simple LSTM pytorch
body: <pre><code>import torch,ipdb
  import torch.autograd as autograd
  import torch.nn as nn
  import torch.nn.functional as F
  import torch.optim as optim
  from torch.autograd import Variable
  
  rnn = nn.LSTM(input_size=10, hidden_size=20, num_layers=2)
  input = Variable(torch.randn(5, 3, 10))
  h0 = Variable(torch.randn(2, 3, 20))
  c0 = Variable(torch.randn(2, 3, 20))
  output, hn = rnn(input, (h0, c0))
  </code></pre>
  
  <p>This is the LSTM example from the docs. I don't know understand the following things:</p>
  
  <ol>
  <li>What is output-size and why is it not specified anywhere?</li>
  <li>Why does the input have 3 dimensions. What does 5 and 3 represent?</li>
  <li>What are 2 and 3 in h0 and c0, what do those represent?</li>
  </ol>
  
  <p>Edit:</p>
  
  <pre><code>import torch,ipdb
  import torch.autograd as autograd
  import torch.nn as nn
  import torch.nn.functional as F
  import torch.optim as optim
  from torch.autograd import Variable
  import torch.nn.functional as F
  
  num_layers=3
  num_hyperparams=4
  batch = 1
  hidden_size = 20
  rnn = nn.LSTM(input_size=num_hyperparams, hidden_size=hidden_size, num_layers=num_layers)
  
  input = Variable(torch.randn(1, batch, num_hyperparams)) # (seq_len, batch, input_size)
  h0 = Variable(torch.randn(num_layers, batch, hidden_size)) # (num_layers, batch, hidden_size)
  c0 = Variable(torch.randn(num_layers, batch, hidden_size))
  output, hn = rnn(input, (h0, c0))
  affine1 = nn.Linear(hidden_size, num_hyperparams)
  
  ipdb.set_trace()
  print output.size()
  print h0.size()
  </code></pre>
  
  <blockquote>
    <p>*** RuntimeError: matrices expected, got 3D, 2D tensors at</p>
  </blockquote>