Thanks it worked.
Out of curiosity, I have one more question
as you see, I am calling .eval(), and .train() correctly prior to each loss.backward().
Therefore, even though I just decided to add retain_graph=True to bypass the warning, the training should converge as if I detached the generated input.
Am I correct here?