When you do loss.backward(), the gradients are calculated for both the networks and stored along with the parameters, assuming you are doing something like net2(net1(input)). Then according to how your optimizer is defined, when you do optimizer.step() the weights will get updated.