import torch
from torch.autograd import Variable
temp_a = torch.rand(1, 4)
a = torch.rand(1, 4)
b = a**2
c = b*2
d = c.mean()
a.copy_(temp_a)
d.backward()

Hi in this toy example, if I replace the original parameters or input of the model (e.g. a) with another set of the parameters (e.g. temp_a). Then I do backward. Is the calculated gradient w.r.t the original input a or the  temp_a.
This is a just a toy example that shows my intention. In my implementation,

I cloned the model parameters at the beginning.
And then I do an intermediate step that would forward, backward, and update the model parameters with autograd.
I forward the model again and get a new output. And I tried to get the gradient of the current output w.r.t the original model parameters I cloned. So I copy the original model parameters back to the model just as I did in the toy example. And I will do the backward and get the gradient.
Finally, I will update the original model parameters with the gradient I get in the last step.

Can this work as I expected? Thank you for your help.