Hi,
I’m trying to train 2 networks in the following scheme:
netA, netB = Net(), Net()
netA_opt = torch.optim.SGD(netA.parameters(), lr=0.1)
netB_opt = torch.optim.SGD(netB.parameters(), lr=0.1)
#Phase 1:
y1_A = netA(X1)
l1 = cretirion1(y,y1_A)
netA_opt.zero_grad()
l1.backward()
#Phase 2:
y2_A = netA(X2)
y2_B = netB(X2)
l2 = cretirion2(y2_A,y2_B)
netB_opt.zero_grad()
l2.backward(retain_graph=True)
netB_opt.step()
#Phase 3:
y1_B = netB(X1)
l3 = cretirion3(y,y1_B)
l3.backward()
netA_opt.step()
Now, in phase 2 netB is updated as a function of netA predictions. In phase 3 I want to optimize netA to give better updates in phase 2, so netB will minimize l3. In practice I see that l3.backward() does not calculate any gradients w.r.t netA. What should I change?
Thanks in advance!