I’m not sure I understand the question correctly. autograd.grad will calculate the gradients as well, but won’t accumulate them in the .grad attributes of the leaf tensors, but will return them instead.