Think you could include a diagram of this setup? And maybe a reference of a similar project? It’s a bit unclear what the goal is.
I don’t think that you can just call .backward() on a summed loss (though I could be wrong) and have it propagate through all three of these networks in order to attribute the correct gradients to the various parameters. But again, it’s hard to tell without a clearer picture of what you are trying to accomplish.
This is a shameless self-reference, but here I was working with a GAN/Autoencoder setup that uses two loss calculations. One for the AE’s reconstruction loss, and one for the D(G(z)) GAN loss. You may also look up the EBGAN paper 1 (Appendix C) which uses an autoencoder as a discriminator.