At the moment I did like this

optimizer.zero_grad()
loss1.backward(retain_graph=True)
loss2.backward()
optimizer.step()

Maybe that’s how it will be right?

optimizer.zero_grad()
loss1.backward(retain_graph=True)
optimizer.step()
optimizer.zero_grad()
loss2.backward()
optimizer.step()