If I update two connected models , loss term become NAN
X = # ...
temp = Model1(X)
out = Model2(temp)
loss = loss_func(out,label)

Model1_optimizer.zero_grad()
Model2_optimizer.zero_grad()

loss.backward()

Model1_optimizer.step()
Model2_optimizer.step()

I also try to use one optimizer which contain 2 models’ weights.
params = list(Model1.parameters()) + list(Model2.parameters())
opt = torch.optim.Adam(params, lr=0.002)

X = # ...
temp = Model1(X)
out = Model2(temp)
loss = loss_func(out,label)

opt.zero_grad()
loss.backward()
opt.step()

I still got NAN loss term.
However, when I update only one of the two, it works well.
But I still want to update both for better performance.
What should I do?