Hi all,
Is there a way to calculate partial derivatives instead of total derivative like tf.gradients with parameter stop_gradients https://www.tensorflow.org/api_docs/python/tf/gradients 1
a = torch.tensor(1., requires_grad=True)
b = 2 * a
c = a + b
print(torch.autograd.grad(c, a, retain_graph=True))

Output (tensor(3.),), Expected output: 1 for partial derivative.
I know we can use b = 2*(a.detach()). But what if d=2*c. In that case, I want to keep calculating the total derivative dd/da. I want to get dd/da=6 without a.detach instead of 2.
Background: I am trying to calculate the last hidden layer partial gradient w.r.t the the first hidden in RNN. Then I still need to calculate loss.backward() where I don’t want to detach the hidden layer.
Thank you all.