Thanks. Just to understand a few things better, Did something underlying with the way autograd works change between torch 1.0 and torch 1.7?
The PyTorch implementation of cluster GAN architecture (torch 1.0) uses the following way to update their networks.
opt1.zero_grad()
loss1.backward(retain_graph=True)
opt1.step()

opt2.zero_grad()
loss2.backward()
opt2.step()

The above doesn’t work in torch 1.7
I am not able to reproduce the results using the following suggested change.
opt1.zero_grad()
opt2.zero_grad()
loss1.backward(retain_graph=True)
loss2.backward()
opt1.step()
opt2.step()

Do you have any insights as to why this could happen?