title: RuntimeError: Found dtype Long but expected Float: When using criterion
body: <p>I realized this question has been asked many times but I cannot find any solution which fixes my issue. I found this code online and tried to run it to understand how it actually works. I found the error is RuntimeError: Found dtype Long but expected Float. It happens at errD_real = criterion(output, label). I tried to add .float() to change the type, but it does not work. I use Pytorch 1.7.0, Python 3.8.3, Anaconda 1.9.12, Cudnn 7.6.0. The error and code are below.</p>
  <pre><code>runfile('C:/Summer2020/computer_vision/final_project/copy_of_chest_x_ray_shenanigans_(dcgan).py', wdir='C:/Summer2020/computer_vision/final_project')
  Generator(
    (main): Sequential(
      (0): ConvTranspose2d(100, 512, kernel_size=(4, 4), stride=(1, 1), bias=False)
      (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU(inplace=True)
      (3): ConvTranspose2d(512, 256, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)
      (4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (5): ReLU(inplace=True)
      (6): ConvTranspose2d(256, 128, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)
      (7): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (8): ReLU(inplace=True)
      (9): ConvTranspose2d(128, 64, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)
      (10): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (11): ReLU(inplace=True)
      (12): ConvTranspose2d(64, 3, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)
      (13): Tanh()
    )
  )
  Discriminator(
    (main): Sequential(
      (0): Conv2d(3, 64, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)
      (1): LeakyReLU(negative_slope=0.2, inplace=True)
      (2): Conv2d(64, 128, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)
      (3): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (4): LeakyReLU(negative_slope=0.2, inplace=True)
      (5): Conv2d(128, 256, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)
      (6): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (7): LeakyReLU(negative_slope=0.2, inplace=True)
      (8): Conv2d(256, 512, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)
      (9): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (10): LeakyReLU(negative_slope=0.2, inplace=True)
      (11): Conv2d(512, 1, kernel_size=(4, 4), stride=(1, 1), bias=False)
      (12): Sigmoid()
    )
  )
  Starting Training Loop...
  Type of output is  &lt;class 'torch.Tensor'&gt;
  Type of label is  &lt;class 'torch.Tensor'&gt;
  Traceback (most recent call last):
  
    File &quot;C:\Summer2020\computer_vision\final_project\copy_of_chest_x_ray_shenanigans_(dcgan).py&quot;, line 337, in &lt;module&gt;
      main()
  
    File &quot;C:\Summer2020\computer_vision\final_project\copy_of_chest_x_ray_shenanigans_(dcgan).py&quot;, line 243, in main
      errD_real = criterion(output, label)
  
    File &quot;C:\Users\binhd\anaconda3\envs\t2\lib\site-packages\torch\nn\modules\module.py&quot;, line 727, in _call_impl
      result = self.forward(*input, **kwargs)
  
    File &quot;C:\Users\binhd\anaconda3\envs\t2\lib\site-packages\torch\nn\modules\loss.py&quot;, line 530, in forward
      return F.binary_cross_entropy(input, target, weight=self.weight, reduction=self.reduction)
  
    File &quot;C:\Users\binhd\anaconda3\envs\t2\lib\site-packages\torch\nn\functional.py&quot;, line 2525, in binary_cross_entropy
      return torch._C._nn.binary_cross_entropy(
  
  RuntimeError: Found dtype Long but expected Float
  </code></pre>
  <pre><code>from __future__ import print_function
  import argparse
  import os
  import random
  import torch
  import torch.nn as nn
  import torch.nn.parallel
  import torch.backends.cudnn as cudnn
  import torch.optim as optim
  import torch.utils.data
  import torchvision.datasets as dset
  import torchvision.transforms as transforms
  import torchvision.utils as vutils
  import numpy as np
  import matplotlib.pyplot as plt
  import matplotlib.animation as animation
  from IPython.display import HTML
  
  def main():
      # Set random seed for reproducibility
      manualSeed = 999
      #manualSeed = random.randint(1, 10000) # use if you want new results
      random.seed(manualSeed)
      torch.manual_seed(manualSeed)
      
      # Load dataset
      
      dataroot = 'C:\Spring 2020/Machine Learning and Computer Vision/Project Database/chest xray/chest_xray/chest_xray/train/'
      
      # Number of workers for dataloader
      workers = 4
      
      # Batch size during training
      batch_size = 128
      
      # Spatial size of training images. All images will be resized to this
      #   size using a transformer.
      image_size = 64
      
      # Number of channels in the training images. For color images this is 3
      nc = 3
      
      # Size of z latent vector (i.e. size of generator input)
      nz = 100
      
      # Size of feature maps in generator
      ngf = 64
      
      # Size of feature maps in discriminator
      ndf = 64
      
      # Number of training epochs
      num_epochs = 5
      
      # Learning rate for optimizers
      lr = 0.0002
      
      # Beta1 hyperparam for Adam optimizers
      beta1 = 0.5
      
      # Number of GPUs available. Use 0 for CPU mode.
      ngpu = 1
      
      # Create the dataset
      dataset = dset.ImageFolder(root='C:\Spring 2020/Machine Learning and Computer Vision/Project Database/chest xray/chest_xray/chest_xray/train/',
                                 transform=transforms.Compose([
                                     transforms.Resize(image_size),
                                    #  transforms.Grayscale(1),
                                     transforms.CenterCrop(image_size),
                                     transforms.ToTensor(),
                                     transforms.Normalize((0.5), (0.5)),
                                 ]))
      # Create the dataloader
      dataloader = torch.utils.data.DataLoader(dataset, batch_size=batch_size,
                                               shuffle=True, num_workers=workers)
      
      # Decide which device we want to run on
      device = torch.device(&quot;cuda:0&quot; if (torch.cuda.is_available() and ngpu &gt; 0) else &quot;cpu&quot;)
      
      # Plot some training images
      real_batch = next(iter(dataloader))
      plt.figure(figsize=(8,8))
      plt.axis(&quot;off&quot;)
      plt.title(&quot;Training Images&quot;)
      plt.imshow(np.transpose(vutils.make_grid(real_batch[0].to(device)[:64], padding=2, normalize=True).cpu(),(1,2,0)))
      
      # Custom weights initialization called on netG and netD
      def weights_init(m):
          classname = m.__class__.__name__
          if classname.find('Conv') != -1:
              nn.init.normal_(m.weight.data, 0.0, 0.02)
          elif classname.find('BatchNorm') != -1:
              nn.init.normal_(m.weight.data, 1.0, 0.02)
              nn.init.constant_(m.bias.data, 0)
      
      &quot;&quot;&quot;Generator Code&quot;&quot;&quot;
      
      class Generator(nn.Module):
          def __init__(self, ngpu):
              super(Generator, self).__init__()
              self.ngpu = ngpu
              self.main = nn.Sequential(
                  # input is Z, going into a convolution
                  nn.ConvTranspose2d( nz, ngf * 8, 4, 1, 0, bias=False),
                  nn.BatchNorm2d(ngf * 8),
                  nn.ReLU(True),
                  # state size. (ngf*8) x 4 x 4
                  nn.ConvTranspose2d(ngf * 8, ngf * 4, 4, 2, 1, bias=False),
                  nn.BatchNorm2d(ngf * 4),
                  nn.ReLU(True),
                  # state size. (ngf*4) x 8 x 8
                  nn.ConvTranspose2d( ngf * 4, ngf * 2, 4, 2, 1, bias=False),
                  nn.BatchNorm2d(ngf * 2),
                  nn.ReLU(True),
                  # state size. (ngf*2) x 16 x 16
                  nn.ConvTranspose2d( ngf * 2, ngf, 4, 2, 1, bias=False),
                  nn.BatchNorm2d(ngf),
                  nn.ReLU(True),
                  # state size. (ngf) x 32 x 32
                  nn.ConvTranspose2d( ngf, nc, 4, 2, 1, bias=False),
                  nn.Tanh()
                  # state size. (nc) x 64 x 64
              )
      
          def forward(self, input):
              return self.main(input)
  
      
      # Create the generator
      netG = Generator(ngpu).to(device)
      
      # Handle multi-gpu if desired
      if (device.type == 'cuda') and (ngpu &gt; 1):
          netG = nn.DataParallel(netG, list(range(ngpu)))
      
      # Apply the weights_init function to randomly initialize all weights
      #  to mean=0, stdev=0.2.
      netG.apply(weights_init)
      
      # Print the model
      print(netG)
      
      &quot;&quot;&quot;Discriminator Code
      
      
      &quot;&quot;&quot;
      
      class Discriminator(nn.Module):
          def __init__(self, ngpu):
              super(Discriminator, self).__init__()
              self.ngpu = ngpu
              self.main = nn.Sequential(
                  # input is (nc) x 64 x 64
                  nn.Conv2d(nc, ndf, 4, 2, 1, bias=False),
                  nn.LeakyReLU(0.2, inplace=True),
                  # state size. (ndf) x 32 x 32
                  nn.Conv2d(ndf, ndf * 2, 4, 2, 1, bias=False),
                  nn.BatchNorm2d(ndf * 2),
                  nn.LeakyReLU(0.2, inplace=True),
                  # state size. (ndf*2) x 16 x 16
                  nn.Conv2d(ndf * 2, ndf * 4, 4, 2, 1, bias=False),
                  nn.BatchNorm2d(ndf * 4),
                  nn.LeakyReLU(0.2, inplace=True),
                  # state size. (ndf*4) x 8 x 8
                  nn.Conv2d(ndf * 4, ndf * 8, 4, 2, 1, bias=False),
                  nn.BatchNorm2d(ndf * 8),
                  nn.LeakyReLU(0.2, inplace=True),
                  # state size. (ndf*8) x 4 x 4
                  nn.Conv2d(ndf * 8, 1, 4, 1, 0, bias=False),
                  nn.Sigmoid()
              )
      
          def forward(self, input):
              return self.main(input)
      
  
      
      # Create the Discriminator
      netD = Discriminator(ngpu).to(device)
      
      # Handle multi-gpu if desired
      if (device.type == 'cuda') and (ngpu &gt; 1):
          netD = nn.DataParallel(netD, list(range(ngpu)))
          
      # Apply the weights_init function to randomly initialize all weights
      #  to mean=0, stdev=0.2.
      netD.apply(weights_init)
      
      # Print the model
      print(netD)
      
      &quot;&quot;&quot;Train the GAN!
      
      &quot;&quot;&quot;
      
      # Initialize BCELoss function
      criterion = nn.BCELoss()
      
      # Create batch of latent vectors that we will use to visualize
      #  the progression of the generator
      fixed_noise = torch.randn(64, nz, 1, 1, device=device)
      
      # Establish convention for real and fake labels during training
      real_label = 1
      fake_label = 0
      
      # Setup Adam optimizers for both G and D
      optimizerD = optim.Adam(netD.parameters(), lr=0.0002, betas=(beta1, 0.999))
      optimizerG = optim.Adam(netG.parameters(), lr=0.001, betas=(beta1, 0.999))
      
      # Commented out IPython magic to ensure Python compatibility.
      # Training Loop
      
      # Lists to keep track of progress
      img_list = []
      G_losses = []
      D_losses = []
      iters = 0
      num_epochs = 35
      
      print(&quot;Starting Training Loop...&quot;)
      # For each epoch
      for epoch in range(num_epochs):
          # For each batch in the dataloader
          for i, data in enumerate(dataloader, 0):
              
              ############################
              # (1) Update D network: maximize log(D(x)) + log(1 - D(G(z)))
              ###########################
              ## Train with all-real batch
              netD.zero_grad()
              # Format batch
              real_cpu = data[0].to(device)
              b_size = real_cpu.size(0)
              label = torch.full((b_size,), real_label, device=device) # 64 = batchSize
              # print(&quot;label &quot; + str(label.size()))
              # Forward pass real batch through D
              output = netD(real_cpu).view(-1) # 256??
              print('Type of output is ', type(output))
              print('Type of label is ', type(label))
              # print(&quot;output &quot; + str(output.size()))
              # Calculate loss on all-real batch
              errD_real = criterion(output, label)
              # Calculate gradients for D in backward pass
              errD_real.backward()
              D_x = output.mean().item()
      
              ## Train with all-fake batch
              # Generate batch of latent vectors
              noise = torch.randn(b_size, nz, 1, 1, device=device)
              # Generate fake image batch with G
              fake = netG(noise)
              # print(&quot;fake &quot; + str(fake.size()))
              label.fill_(fake_label)
              # Classify all fake batch with D
              output = netD(fake.detach()).view(-1)
              # Calculate D's loss on the all-fake batch
              errD_fake = criterion(output, label)
              # Calculate the gradients for this batch
              errD_fake.backward()
              D_G_z1 = output.mean().item()
              # Add the gradients from the all-real and all-fake batches
              errD = errD_real + errD_fake
              # Update D
              optimizerD.step()
      
              ############################
              # (2) Update G network: maximize log(D(G(z)))
              ###########################
              netG.zero_grad()
              label.fill_(real_label)  # fake labels are real for generator cost
              # Since we just updated D, perform another forward pass of all-fake batch through D
              output = netD(fake).view(-1)
              # Calculate G's loss based on this output
              errG = criterion(output, label)
              # Calculate gradients for G
              errG.backward()
              D_G_z2 = output.mean().item()
              # Update G
              optimizerG.step()
              
              # Output training stats
              if i % 50 == 0:
                  print('[%d/%d][%d/%d]\tLoss_D: %.4f\tLoss_G: %.4f\tD(x): %.4f\tD(G(z)): %.4f / %.4f'
                         % (epoch, num_epochs, i, len(dataloader),
                           errD.item(), errG.item(), D_x, D_G_z1, D_G_z2))
              
              # Save Losses for plotting later
              G_losses.append(errG.item())
              D_losses.append(errD.item())
              
              # Check how the generator is doing by saving G's output on fixed_noise
              if (iters % 500 == 0) or ((epoch == num_epochs-1) and (i == len(dataloader)-1)):
                  with torch.no_grad():
                      fake = netG(fixed_noise).detach().cpu()
                  img_list.append(vutils.make_grid(fake, padding=2, normalize=True))
                  
              iters += 1
      
  
      
      plt.figure(figsize=(10,5))
      plt.title(&quot;Generator and Discriminator Loss During Training&quot;)
      plt.plot(G_losses,label=&quot;G&quot;)
      plt.plot(D_losses,label=&quot;D&quot;)
      plt.xlabel(&quot;iterations&quot;)
      plt.ylabel(&quot;Loss&quot;)
      plt.legend()
      plt.show()
  
      fig = plt.figure(figsize=(8,8))
      plt.axis(&quot;off&quot;)
      ims = [[plt.imshow(np.transpose(i,(1,2,0)), animated=True)] for i in img_list]
      ani = animation.ArtistAnimation(fig, ims, interval=1000, repeat_delay=1000, blit=True)
      
      HTML(ani.to_jshtml())
   
      # Grab a batch of real images from the dataloader
      real_batch = next(iter(dataloader))
      
      # Plot the real images
      plt.figure(figsize=(80,80))
      plt.subplot(1,2,1)
      plt.axis(&quot;off&quot;)
      plt.title(&quot;Real Images&quot;)
      plt.imshow(np.transpose(vutils.make_grid(real_batch[0].to(device)[:64], padding=5, normalize=True).cpu(),(1,2,0)))
      
      # Plot the fake images from the last epoch
      plt.subplot(1,2,2)
      plt.axis(&quot;off&quot;)
      plt.title(&quot;Fake Images&quot;)
      plt.imshow(np.transpose(img_list[-1],(1,2,0)))
      plt.show()
  
  
  if __name__ == '__main__':
      main()
  </code></pre>