title: Derive the gradient through torch.topk
body: <p>I want to derive the gradient through <code>torch.topk</code> function.</p>
  <p>Suppose I the input is a vector
  <a href="https://i.stack.imgur.com/q3wno.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/q3wno.png" alt="enter image description here" /></a>,</p>
  <p>then it is transformed by a parameter matrix
  <a href="https://i.stack.imgur.com/JeoQA.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/JeoQA.png" alt="enter image description here" /></a> ,</p>
  <p>and the top k values of the vector are selected
  <a href="https://i.stack.imgur.com/vxF90.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/vxF90.png" alt="enter image description here" /></a>.</p>
  <p>The result vector is further transformed by <strong>element-wise multiplication</strong>.
  <a href="https://i.stack.imgur.com/z7OF1.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/z7OF1.png" alt="enter image description here" /></a></p>
  <p>Finally the loss is computed by</p>
  <p><a href="https://i.stack.imgur.com/EqkE5.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/EqkE5.png" alt="enter image description here" /></a>.</p>
  <p>I wonder, is the loss differentiable with respect to W? Formally, can we calculate the following gradient?</p>
  <p><a href="https://i.stack.imgur.com/7TW8K.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/7TW8K.png" alt="enter image description here" /></a></p>