title: Pytorch. How does pin_memory work in Dataloader?
body: <p>I want to understand how pin_memory in Dataloader works.</p>
  <p>According to the documentation:</p>
  <pre><code>pin_memory (bool, optional) – If True, the data loader will copy tensors into CUDA pinned memory before returning them.
  </code></pre>
  <p>Below is a self-contained code example.</p>
  <pre><code>import torchvision
  import torch
  
  print('torch.cuda.is_available()', torch.cuda.is_available())
  train_dataset = torchvision.datasets.CIFAR10(root='cifar10_pytorch', download=True, transform=torchvision.transforms.ToTensor())
  train_dataloader = torch.utils.data.DataLoader(train_dataset, batch_size=64, pin_memory=True)
  x, y = next(iter(train_dataloader))
  print('x.device', x.device)
  print('y.device', y.device)
  </code></pre>
  <p>Producing the following output:</p>
  <pre><code>torch.cuda.is_available() True
  x.device cpu
  y.device cpu
  </code></pre>
  <p>But I was expecting something like this, because I specified flag <code>pin_memory=True</code> in <code>Dataloader</code>.</p>
  <pre><code>torch.cuda.is_available() True
  x.device cuda:0
  y.device cuda:0
  </code></pre>
  <p>Also I run some benchmark:</p>
  <pre><code>import torchvision
  import torch
  import time
  import numpy as np
  
  pin_memory=True
  train_dataset =torchvision.datasets.CIFAR10(root='cifar10_pytorch', download=True, transform=torchvision.transforms.ToTensor())
  train_dataloader = torch.utils.data.DataLoader(train_dataset, batch_size=64, pin_memory=pin_memory)
  print('pin_memory:', pin_memory)
  times = []
  n_runs = 10
  for i in range(n_runs):
      st = time.time()
      for bx, by in train_dataloader:
          bx, by = bx.cuda(), by.cuda()
      times.append(time.time() - st)
  print('average time:', np.mean(times))
  </code></pre>
  <p>I got the following results.</p>
  <pre><code>pin_memory: False
  average time: 6.5701503753662
  
  pin_memory: True
  average time: 7.0254474401474
  </code></pre>
  <p>So <code>pin_memory=True</code> only makes things slower.
  Can someone explain me this behaviour?</p>