Right, so I have two loss.backwards() but still one forward.
My question is if changing the order of G/D update gives the same result.
(e.g. opt_G.step() updates G, and loss_D depends on G, so loss_D.backward() calculates the grad based on the new G which is wrong?)