Thanks Ptrblck, but I still not able to fix the bug. I am trying to reproduce the codes here: https://github.com/whxbergkamp/RobustDL_GAN/blob/master/cifar10/adversarial_networks/train_gan_cifar10.py 3 , where their workflow is like this:
G_optimizer1
G_optimizer2
G_optimizer3
G_optimizer4
G_optimizer5
G_virtual_optimizer (forward)
D_optimizer
G_virtual_optimizer (backward)
They are using tensorflow and they can manipulate the gradients directly. But in pytorch, I need to do optimizer.step() for G_virtual_optimizer (forward),
D_optimizer and G_virtual_optimizer (backward) based on G_virtual_loss and D_loss, where
G_virtual_loss(forward) = g_loss6 = -1*criterion(d_out_noise6, labels);
G_virtual_loss(backward) = -1 * g_loss6 = criterion(d_out_noise6, labels);
D_loss = d_loss_real + d_loss6 + d_loss_noiseB = d_loss_real + criterion(d_out_noise6, labels) + d_loss_noiseB.
The thing is that g_loss6 and d_loss_noise6 share the same variable – d_out_noise6. I tried to use g_loss6_copy = g_loss.detach() so that I can update G_virtual_optimizer (forward), D_optimizer and G_virtual_optimizer (backward) successfully. However, if I use detach(), the gradients explode very quickly. So, now I am switching back to PyTorch 1.4 to bypass this issue.
My questions are:

if I am doing the updates in PyTorch 1.4, does it exactly reproduce the workflow as theirs in TensorFlow?
if not, what’s the difference and how should I achieve the workflow?