I have simple GAN so I do following.
real_target = 1s
fake_target = 0s

gen_net.train()
dis_net.eval()
gen_optimizer.zero_grad()

generated = gen_net(noise)
gen_loss = loss_fn(dis_net(generated), real_target)

gen_loss.backward()

gen_net.eval()
dis_net.train()
dis_optimizer.zero_grad()

input_data = torch.cat(generated, real)
target = torch.cat(fake_target, greal_target)

output = dis_net(input_data)
dis_loss = loss_fn(output, target)

dis_loss.backward()

Yes, I am aware of the fact that once second backward() is called, the framework will attempt to backprop through generated, gen_net and dis_net twice and therefore raise warning.
Based on the code I have above, my understanding is that gen_net will be updated only from the first backward and dis_net will be updated only from the second backward as I call .eval() and .train() correctly prior to forward pass.
So, if I am safe to set retain_graph=True to get the training I want. Is my understanding correct?
Furthermore, I would like to deal with this correctly.
I believe if code is written correctly, I should not see retain_graph=True warning at all.
How should I change the above code so that I do not see retain_graph=True warning without explicitly setting retain_graph=True?