title: PyTorch Autograd Differentiated Tensors appears to not have been used in the graph
body: <p>I'm trying to improve a CNN I made by implementing a weighted loss method described in <a href="https://arxiv.org/pdf/1803.09050.pdf" rel="noreferrer">this paper</a>. To do this, I looked into <a href="https://github.com/danieltan07/learning-to-reweight-examples/blob/master/Learning%20to%20Reweight%20Examples%20for%20Robust%20Deep%20Learning.ipynb" rel="noreferrer">this notebook</a> which implements the pseudo-code of the method described in the paper.</p>
  <p>When translating their code to my model, I ran into the error <code>RuntimeError: One of the differentiated Tensors appears to not have been used in the graph. Set allow_unused=True if this is the desired behavior</code> when using <code>torch.autograd.grad()</code>.</p>
  <p>My code and the error is in the second to last line:</p>
  <pre><code>for epoch in range(1): #tqdm(range(params['epochs'])):
    model.train()
    text_t, labels_t = next(iter(train_iterator))
    text_t = to_var(text_t, requires_grad=False)
    labels_t = to_var(labels_t, requires_grad=False)
    dummy = L2RWCNN(INPUT_DIM, EMBEDDING_DIM, N_FILTERS, FILTER_SIZES, OUTPUT_DIM, 
                  DROPOUT, PAD_IDX)
    dummy.state_dict(model.state_dict())
    dummy.cuda()
    y_f_hat = dummy(text_t)
    cost = F.binary_cross_entropy_with_logits(y_f_hat.squeeze(), labels_t, reduce = False)
    eps = to_var(torch.zeros(cost.size()))
    l_f_meta = torch.sum(cost * eps)
    dummy.zero_grad()
    num_params = 0
    grads = torch.autograd.grad(l_f_meta, (dummy.params()), create_graph = True)
    with torch.no_grad():
      for p, grad in zip(dummy.parameters(), grads):
        tmp = p - params['lr'] * grad
        p.copy_(tmp)
    text_v, labels_v = next(iter(valid_iterator))
    y_g_hat = dummy(text_v)
    l_g_meta = F.binary_cross_entropy_with_logits(y_g_hat.squeeze(), labels_v, reduce = False)
    l_g_meta = torch.sum(l_g_meta)
    grad_eps = torch.autograd.grad(l_g_meta, eps, only_inputs=True)[0]
    print(grad_eps)
  </code></pre>
  <p>I think the error is because <code>eps</code> was not in any previous <code>torch.autograd.grad()</code> calls. I tried the suggested solution of setting <code>allow_unused=True</code> but that resulted in a <code>None</code> value. I looked at <a href="https://stackoverflow.com/questions/54763081/using-autograd-grad-as-a-parameter-for-a-loss-function-pytorch">this post</a> to find a solution, but the method that fixed the problem here (don't slice the tensors) doesn't work for me because I'm not passing in any partial variables. I also tried setting <code>create_graph = False</code> in my first <code>autograd.grad()</code> call, but that didn't fix the issue. Does anyone have a solution?</p>
  <p><a href="https://colab.research.google.com/drive/1aY7XlgUClEe5Ldeu8DjSsxuj9sXTtRw4?usp=sharing" rel="noreferrer">My full code if that's needed</a></p>
  <p>EDIT:
  Created a new post with a different phrasing to the question <a href="https://stackoverflow.com/questions/67008171/pytorch-backwards-automatic-diff-with-weighted-loss">here</a></p>