Hello,
In this resource https://github.com/seangal/dcgan_vae_pytorch/blob/master/main.py 192
it shows how to train a VaE+GAN architecture, by training the discriminator on noise and real data, and the generator on reconstructed data (which have to fool the discriminator).
My problem is that when I run the code on a face dataset, although the discriminator can tell between noise-generated and real examples, it completely fails to detect the reconstructed examples as fake. If I stop training the generator, the discriminator can tell the difference. However it seems with the generator being trained, it may be creating adversarial examples against the discriminator which still look like random noise.
Would it theoretically be a mistake to train the discriminator on both noise generated examples and reconstructed examples?