Hi @smth, thanks for the reply.
I have been seeing  some strange behavior (mostly good).
I am calling optimizer1.step() after I do loss1.backward() and optimizer2.step() after loss2.bakward(), so the double gradient shouldn’t affect anything since I already did the associated optimizer step. Is that correct?
Since loss1 and loss2 are associated with different models (no shared params), does the lambda*loss1 make any difference to loss2? I was starting to think that it doesn’t, but the two models (case 1 and case 2) give quite different results.
Thanks,
Gautam