Hey,
Thanks for your response. I am just a bit confused. So I have following code structure:
# D & G both are CNNs
# opt_D optimizer object for D.parameters() & opt_G is for G.parameters()

# To update D
D.train()
opt_D.zero_grad()

# sample x,y,z from data set 
with torch.no_grad(): # 1) As do not want to update G 
     fake = G(x,y) 

d_out_fake = D(x, fake) # 2) not sure if it is possible to fake.detach()
d_out_real = D(x, z)
d_loss = loss_fn1(d_out_fake, d_out_real)
d_loss.backward()
opt_D.step()


# To update G
G.train()
opt_G.zero_grad()

# sample x,y,z from data set 

fake = G(x,y) 
# 3) As do not want to update D
with torch.no_grad(): 
    d_out_fake = D(x, fake) 
    d_out_real = D(x, z)

g_loss = loss_fn2(d_out_fake, d_out_real)
g_loss.backward()
opt_G.step()

As you see there are separate optimizers for both of the models. Also, I read no_grad() don’t let gradients to be calculated. In my opinion I think I do not need to calculate grads on D while updating G but a bit puzzled currently. Would be great to have your insights.