This might be a very silly question, but to get the gradients of parameters for different losses (as in the question) don’t we have to subtract the gradients for the second update? I’m basing this on the fact that we don’t do an optim.zero_grad() in between (which will anyway lead to wrong back propagation as suggested). To be clear, shouldn’t the below code be used rather than what the question suggests?
opt.zero_grad()

loss1.backward()
grad_wrt_loss1 = next(net.parameters()).grad
print(grad_wrt_loss1)
loss2.backward()
# THIS subtraction is the distinction from above code:
grad_wrt_loss2 = next(net.parameters()).grad - grad_wrt_loss1
print(grad_wrt_loss2)

opt.step()