What is the difference (theoretically) between using 2 different optimizers for two networks (for example 2 generators/discriminators) and using 1 in which we sum the losses of both networks and bp it?
For example in CycleGAN, there are two optimizers for the two Ds and one for the two Gs.