in case 2, you dont need to do loss1.backward(). Because loss2 = loss2 + lambda * loss1, when you call loss2.backward(), loss1’s backward will also be called.
You can use retain_variables=True, which will keep all the intermediate buffers around to do loss1.backward twice, but it’ll lead in double gradients in your case.