title: RuntimeError: Expected hidden[0] size (1, 1, 512), got (1, 128, 512) for LSTM pytorch
body: <p>I trained the LSTM with a batch size of 128 and during testing my batch size is 1, why do I get this error? I'm suppose to initialize the hidden size when doing testing?</p>
  <p>Here is the code that i'm using, I initialize the hidden state <strong>init_hidden</strong> function as <em>(number_of_layers, batch_size, hidden_size)</em> since <em>batch_first=True</em></p>
  <pre><code>class ImageLSTM(nn.Module):
  def __init__(self,  n_inputs:int=49, 
                      n_outputs:int=4096, 
                      n_hidden:int=256, 
                      n_layers:int=1, 
                      bidirectional:bool=False):
      &quot;&quot;&quot;
      Takes a 1D flatten images.
      &quot;&quot;&quot;
      super(ImageLSTM, self).__init__()
      self.n_inputs   = n_inputs
      self.n_hidden   = n_hidden
      self.n_outputs  = n_outputs
      self.n_layers   = n_layers
      self.bidirectional = bidirectional
      self.lstm       = nn.LSTM(  input_size=self.n_inputs, 
                                  hidden_size=self.n_hidden, 
                                  num_layers=self.n_layers, 
                                  dropout  = 0.5 if self.n_layers&gt;1 else 0,
                                  bidirectional=self.bidirectional,
                                  batch_first=True)
      if (self.bidirectional):
          self.FC         = nn.Sequential(
                                          nn.Linear(self.n_hidden*2, self.n_outputs),
                                          nn.Dropout(p=0.5),
                                          nn.Sigmoid()
                                              )
  
      else:
          self.FC         = nn.Sequential(
                                          nn.Linear(self.n_hidden, self.n_outputs),
                                          # nn.Dropout(p=0.5),
                                          nn.Sigmoid()
                                          )
  
      
  def init_hidden(self, batch_size, device=None): # input 4D tensor: (batch size, channels, width, height)
      # initialize the hidden and cell state to zero
      # vectors:(number of layer, batch size, number of hidden nodes)
      if (self.bidirectional):
          h0 = torch.zeros(2*self.n_layers,  batch_size, self.n_hidden)
          c0 = torch.zeros(2*self.n_layers,  batch_size, self.n_hidden)
      else:
          h0 = torch.zeros(self.n_layers,  batch_size, self.n_hidden)
          c0 = torch.zeros(self.n_layers,  batch_size, self.n_hidden)
  
      if device is not None:
          h0 = h0.to(device)
          c0 = c0.to(device)
      self.hidden = (h0,c0)
  
  def forward(self, X): # X: tensor of shape (batch_size, channels, width, height)
      # forward propagate LSTM
      lstm_out, self.hidden = self.lstm(X, self.hidden) # lstm_out: tensor of shape (batch_size, seq_length, hidden_size)
      # Decode the hidden state of the last time step 
      out = self.FC(lstm_out[:, -1, :])
      return out
  </code></pre>