title: Why doesn't pytorch allow inplace operations on leaf variables?
body: <p>So if I run this code in Pytorch:</p>
  
  <pre><code>x = torch.ones(2,2, requires_grad=True)
  x.add_(1)
  </code></pre>
  
  <p>I will get the error:</p>
  
  <blockquote>
    <p>RuntimeError: a leaf Variable that requires grad is being used in an in-place operation.</p>
  </blockquote>
  
  <p>I understand that Pytorch does not allow inplace operations on leaf variables and I also know that there are ways to get around this restrictions. What I don't understand is the philosophy behind this rule. Why is it wrong to change a leaf variable with inplace operations?</p>