Yes, but what about the second call to backward function for optimizing the generator? my understanding is that there would be gradients from the first call (when optimizing the discriminator) and these are now added to the gradients of the second call …  i.e., the generator would be optimized by gradients of the second backward call (the correct gradients) + the gradients from the first call (when the generator was detached).
If that’s true, can we apply “zero_grad” to the discriminator before using it to optimize the generator?