They’re not completely the same. If you use some optimizers with momentum, the parameters can be even updated if they are not actually used (by the momentum term). Also it matters whether you detach the tensors before forwarding them on your module for generator and discriminator.