If you don’t want to redo the forward, you will have to delay the optimizer step.
You may be able to stash the current gradient into a temporary list (cloning them) for each backward you do. Then you can do all the steps when all the gradients are computed.