title: PyTorch RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same
body: <p>I've found a lot of answers on this topic but none of them helped.</p>
  <p><strong>The error is</strong>:</p>
  <pre><code>RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same
  </code></pre>
  <p><strong>The training loop</strong>:</p>
  <pre class="lang-py prettyprint-override"><code>model = BrainModel()
  model.to(device)
  loss_function = nn.BCELoss()
  optimizer = torch.optim.Adam(model.parameters(), lr=0.01)
  
  for epoch in range(EPOCHS):
      for sequences, labels in train_dataloader:
          optimizer.zero_grad()
          labels = labels.view(BATCH_SIZE, -1)
          sequences, labels = sequences.to(device), labels.view(BATCH_SIZE, -1).to(device)
          print(next(model.parameters()).is_cuda, sequences.get_device(), labels.get_device())
          out = model(sequences) # ERROR HERE
          out, labels = out.type(torch.FloatTensor), labels.type(torch.FloatTensor)
          loss = loss_function(out, labels)
          loss.backward()
          optimizer.step()
  </code></pre>
  <p>You can see one print inside the loop, and its output is:</p>
  <pre><code>True 0 0
  </code></pre>
  <p>which means that all - the model, the x and y - are on cuda. The same code works well when I use CPU but not GPU. I do not understand what else I need to move to <code>device</code>. I always write it the way I did here and it always worked fine :C</p>