I’m currently implementing SAGAN in pytorch, which uses the new nn.utils.spectral_norm (and batchnorm) for normalization. The results appear well during sampling in training, however when I load a snapshot and set the network to eval mode, I get complete garbage as output. If I don’t set eval mode, the first sample after loading is normal, but subsequent samples are distorted, even if I set requires_grad to false on all the parameters. Any ideas how I can fix this?