Solved by _eigenfoo in post #2 
              
              
                Instead of having one loss function and one optimizer for both your discriminator and generator, you could have a loss and optimizer for each of them. This means your training loop would look something like this: 
for i in range(NUM_ITERATIONS):
    loss_gen = ...
    loss_gen.backward()
    optim_g…