Solved by atakanokan in post #3 
              
              
                Thanks for the answer @albanD! 
I realized that since it is the Discriminator training step (netD), I think it should look like this to have gradients till netD but not netG: 
with torch.no_grad():
      y = netG(noise)
f_enc_Y_D, f_dec_Y_D = netD(y)