title: PyTorch Linear Algebra Gradients
body: <p>I'm looking to back-propagate gradients through a singular value decomposition for regularisation purposes. PyTorch currently does not support backpropagation through a singular value decomposition.</p>
  <p>I know that I could write my own custom function that operates on a Variable; takes its .data tensor, applies the torch.svd to it, wraps a Variable around its singular values and returns it in the forward pass, and in the backward pass applies the appropriate Jacobian matrix to the incoming gradients.</p>
  <p>However, I was wondering whether there was a more elegant (and potentially faster) solution, where I could overwrite the &quot;Type Variable doesn't implement stateless method svd&quot; Error directly, call Lapack, etc. ?</p>
  <p>If someone could guide me through the appropriate steps and source files I need to look at, I'd be very grateful. I suppose these steps would similarly apply to other linear algebra operations which have no associated backward method currently.</p>