Solved by ptrblck in post #2 
              
              
                Both approaches will yield the same result, but the method using two losses is wasteful, since you need to call the backward method twice. 
Here is a small code snippet to demonstrate the behavior: 
# Setup
torch.manual_seed(2809)

modelA = nn.Linear(1, 1)
modelB = nn.Linear(1, 1)

criterion = nn.MS…