It’s possible to do the backward with two networks. You have to create a loss function before you can do that though. Once you’ve calculated the loss you can do my_loss.backward()
See the pytorch examples 8 for how it’s done with one network. It’s pretty much the same with two networks