title: How to do backprop in Pytorch (autograd.backward(loss) vs loss.backward()) and where to set requires_grad=True?
body: <p>I have been using Pytorch for a while now. One question I had regarding backprop is as follows:</p>
  
  <p>let's say we have a loss function for a neural network. For doing backprop, I have seen two different versions. One like:</p>
  
  <pre><code>optimizer.zero_grad()
  autograd.backward(loss)
  optimizer.step()
  </code></pre>
  
  <p>and the other one like:</p>
  
  <pre><code>optimizer.zero_grad()
  loss.backward()
  optimizer.step()
  </code></pre>
  
  <p>Which one should I use? Is there any difference between these two versions?</p>
  
  <p>As a last question, do we need to specify the <code>requires_grad=True</code> for the parameters of every layer of our network to make sure their gradients is being computed in the backprop?</p>
  
  <p>For example do I need to specify it for the layer <code>nn.Linear(hidden_size, output_size)</code> inside my network or it is automatically being set to True by default?</p>