title: Difference between 1 LSTM with num_layers = 2 and 2 LSTMs in pytorch
body: <p>I am new to deep learning and currently working on using LSTMs for language modeling. I was looking at the pytorch documentation and was confused by it.</p>
  
  <p>If I create a </p>
  
  <pre><code>nn.LSTM(input_size, hidden_size, num_layers) 
  </code></pre>
  
  <p>where hidden_size = 4 and num_layers = 2, I think I will have an architecture something like:</p>
  
  <pre><code>op0    op1 ....
  LSTM -&gt; LSTM -&gt; h3
  LSTM -&gt; LSTM -&gt; h2
  LSTM -&gt; LSTM -&gt; h1
  LSTM -&gt; LSTM -&gt; h0
  x0     x1 .....
  </code></pre>
  
  <p>If I do something like</p>
  
  <pre><code>nn.LSTM(input_size, hidden_size, 1)
  nn.LSTM(input_size, hidden_size, 1)
  </code></pre>
  
  <p>I think the network architecture will look exactly like above. Am I wrong? And if yes, what is the difference between these two?</p>