Thank you so much for the help~
I have one last question. If I do want to calculate the gradient of a single parameter in a large model, would torch.autograd be my best bet (in terms of performance), or is there better method? I am trying to do something like feature visualization of CNN, where you update the input image but fixing the parameter of a pretrained CNN model.