title: How Batch learning in Pytorch is performed?
body: <p>When you look at how network architecture is built inside the pytorch code, we need to extend the <code>torch.nn.Module</code> and inside <code>__init__</code>, we define the module of networks and pytorch is going to track the gradients of parameters of these modules. Then inside the <code>forward</code> function, we define how the forward pass should be done for our network. </p>
  
  <p>The thing I do not understand here is how the batch learning is going to occur. In none of the definition above including the <code>forward</code> function, we do not care about the dimension of batch of the input to our network. The only thing we need to set to perform batch learning is to add an extra dimension to the input which corresponds to the batch size but nothing inside the network definition is going to be changed if we are working with batch learning. At least, this is the thing I have seen in the codes <a href="https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html#sphx-glr-beginner-blitz-cifar10-tutorial-py" rel="noreferrer">here</a>.</p>
  
  <p>So, if all the things I have explained so far is correct (I would really appreciate if you let me know if I have misunderstood something), how batch learning is performed if nothing is declared regarding the batch size inside the definition of our network class (the class that inherits <code>torch.nn.Module</code>)? Specifically, I am interested to know how batch gradient descent algorithm is implemented in pytorch when we just set <code>nn.MSELoss</code> with batch dimension.</p>