Hi,
I think you have two issues here:

The error you see happens because when you do optimizer.step() in the first iterations, you modify the weights of net inplace. But the final backward would actually need the original values to be able to backprop though the original net. Hence the error you see.
The final backward will backprop in the same net as the first ones so you need the first ones to set retain_graph=True so that the last backward won’t fail saying that buffers have been freed.

The question I would have is: do you want to compute the gradients for the final loss based on the original parameter values or the updated ones?
If you want the original, then you will have to use different version of the net so that you can keep the original values around to be able to backprop to them.
If you want to get wrt the updated ones, you should re-do the forward based on the final parameter values to compute the summed output at the end.