Hi,
Thank you for your reply @albanD.
I am currently using retain_graph and compute the gradient wrt the last updated weights, however, the issue is that summed_output needs to be detached from the original graph since I already made several optimization steps and zeroed the gradients at each step. what is happening now, is that I detach it, then compute output and add it summed_output so that when I do backward, it is done wrt to the last forward pass. This however, is not what I want and I do not think it’s valid.  Per the second option, is this how you considered doing it?
I would like to calculate gradients based off the original parameters (from every iteration). As you noted that I have to use a different version of the net, how can I do that?