title: Why torch does train using mini-batches?
body: <p>I'm currently tring to understand how to train models via <code>pytorch</code>. And while this I saw a pretty interesting feature: passing to train data --- is a mini-batch. For ex.</p>
  <p>There is a code fragment from official pytorch <a href="https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html" rel="nofollow noreferrer">web-site</a></p>
  <pre class="lang-py prettyprint-override"><code>...
  
  data_dir = 'data/hymenoptera_data'
  image_datasets = {x: datasets.ImageFolder(os.path.join(data_dir, x),
                                            data_transforms[x])
                    for x in ['train', 'val']}
  dataloaders = {x: torch.utils.data.DataLoader(image_datasets[x], batch_size=4,
                                               shuffle=True, num_workers=4)
                for x in ['train', 'val']}
  
  ...
  </code></pre>
  <pre class="lang-py prettyprint-override"><code>...
  
  for inputs, labels in dataloaders[phase]:
      inputs = inputs.to(device)
      labels = labels.to(device)
  
      optimizer.zero_grad()
  
      with torch.set_grad_enabled(phase == 'train'):
          outputs = model(inputs)
          _, preds = torch.max(outputs, 1)
          loss = criterion(outputs, labels)
  
  ...
  </code></pre>
  <p>According to this code, inputs that passing to model is a mini-batch.</p>
  <p>I've tied to fiend some info about this, but unsuccessfully.</p>
  <p>But I'm really curious, is it some kind of boost (parallel run, etc.) or neccussary thing. So, would you mind to help me figure it out and tell me, why there is a mini-batch passing to train function?</p>
  <p>N.B. Will not refuse a link to the paper [smily :)].</p>