Hi,
The trick we use is to store with the Tensor enough information to know how it was created (to be able to compute the gradients).
So the loss1 “knows” that it was computed based on the parameters from net1 and so when you call .backward, it will compute the gradients wrt to these parameters. (whatever you do on the side with other nets).