Hi @colesbury, thanks for your illustration.
I have one more question. Lets say I want to backprop “loss3” into “net1” and do not want to backprop “loss 2” to “net2”. In that case I should not write

loss2.backward()
optimizer2.step().

I should only write

loss3 = loss1 + loss2
loss3.backward().             RIGHT ??

In case I have written loss2.backward() and have not written  optimizer2.step(), will that affect my gradients when I compute loss3.backward(). ???