title: calculate gradient of A * relu(A * X * W0) * W1 w.r.t. A
body: <p>I'm trying to manually calculate the gradient to do backpropagation for my neural network</p>
  <p>the formula for my forward function is  <code>A * relu(A * X * W0) * W1</code></p>
  <p>all A, X, W0, W1 are matrices and I want to get the gradient w.r.t A</p>
  <p>I'm using pytorch so it would be great if anyone can show how to get the gradient of this function in pytorch( without using autograd).</p>
  <p>Thanks!</p>