Solved by tom in post #2 
              
              
                You could dofor p in XXX.parameters(): p.requires_grad_(False) for the bits you are not training and set them to True for the bits you are training. Most GAN examples do that, it saves getting and storing unneeded gradients.
You could have g_loss1 and g_loss2 for each discriminator and g_loss = alp…