A follow-up. I was suggested that the discriminator might be overfitting. Indeed, it seems to be the case.
I evaluated separately the cross entropy loss of the discriminator on the positive samples (coming the train split of the dataset), on the negative samples (generated by the generator) and on a validation positive samples (coming from the validation split of the dataset) :



I interpret this as the discriminator being overspecialized on the training set positive samples considering everything else as a negative (like a one class classifier) and therefore I suppose this is a failure since the discriminator would train the generator to generate only the samples that are exactly (or almost) similar to the samples from the training set.
To be honest, I have the feeling this is not exactly what is happening as interpolating in the latent space produces quite diverse images and, by the way, the generator loss is pretty high. For completeness, the generator loss and some generated samples are displayed below; Even though the generator loss is high , I feel like the samples are again not that bad.


For complete completeness, this is generated with GitHub - jeremyfix/gan_experiments , tag v2, and running python3 main.py train --dataset SVHN --dropout 0.0 --wdecay 0.0
In the next post, soon, I will post some results when regularizing the discriminator.