title: Understanding multi-layer LSTM
body: <p>I'm trying to understand and implement multi-layer LSTM. The problem is i don't know how they connect. I'm having two thoughs in mind:</p>
  
  <ol>
  <li><p>At each timestep, the hidden state H of the first LSTM will become the input of the second LSTM.</p></li>
  <li><p>At each timestep, the hidden state H of the first LSTM will become the initial value for the hidden state of the sencond LSTM, and the input of the first LSTM will become the input for the second LSTM.</p></li>
  </ol>
  
  <p>Please help!</p>