@ptrblck  can you please give me your input here:
what happens if i do this:
modelA = nn.Linear(10, 10)
modelB = nn.Linear(10, 10)

x = torch.randn(1, 10)
target = torch.randn(1, 10)

criterionA = nn.MSELoss()
criterionB = nn.L1Loss()

optimizerA = optim.Adam(modelA.parameters(), lr=1e-3)
optimizerB = optim.Adam(modelB.parameters(), lr=1e-3)

y = modelA(x)
z = modelB(y)

# There is no lossA
lossB = criterionB(z, target)


optimizerA.zero_grad()
optimizerB.zero_grad()


# modelB Update
lossB.backward(retain_graph=True)


# Update modelB (lossB grad)
optimizerB.step()

# Can I update model A in this way?
optimizerA.step()

I dont have lossA any more, but im doing optimizerB.step() and  optimizerA.step(). can i updaqte modelA and modelB in this way?