With BatchNorm in eval mode and everything else in training mode, the result is even weirder: The first run also still works, but instead of being distorted, subsequent samples are complete garbage.

Screenshot-2018-09-30_09-20-33.png512×514 86 KB

Setting all the spectral norm to eval, however, produced garbage from the first sample.
I have spectral norm in both the discriminator and generator. I don’t even have an optimizer, as this just loads the weights trained separately and generates samples.