Thank you for the reply, and sorry for the late reply.
Yes, this is my use case where the “old” parameter is updated based on the gradients w.r.t another set of parameters.
I also checked other threads on the forum, where some people said the intermediary results won’t be changed with the model parameters. Is this the case?