I’m training a GAN on a single ImageNet class (~1300 images) and I found that the image normalization constants I pass to torch.transforms.Normalize seem to affect the training a lot.
For example, the following two images are obtained by the generator, after approximately the same number of epochs, on the “panda” class, using mean = (0.485, 0.456, 0.406) and std=(0.229, 0.224, 0.225) for the left image, and mean = std = (0.5, 0.5, 0.5) for the right image.
 
I haven’t run the training for long enough to exclude that the “left” model will eventually improve, but still I find this to be a significant difference for what I thought was a relatively unimportant parameter.
Here is a script which (hopefully) reproduces the error: https://gist.github.com/simopal6/c6484df00d5747dfe33f7ed67383c6fd 17
By default it uses the “good” normalization constants, unless you pass the --bad flag. Also, it needs the dataset, in a directory structure suitable for ImageFolder. You can use any ImageNet folder, though my results were obtained on the “panda” (n02510455) class. For convenience, I uploaded that class on Dropbox at this link: https://www.dropbox.com/s/7kdjz99zqlkp2jb/dataset.zip?dl=0 4 (I’m not sure I’m allowed to do that, but whatever).
P.S. the model is actually a Wasserstein GAN