You could have the net predict an entire distribution for each pixel. Like a mixture of gaussians or something. But then sampling from it would be incorrect. E.g. it might not know if the car is blue or red, so half the pixels would randomly be red, and the other have blue. It would look terrible.<p>Somewhere, the neural net needs to decide &quot;this car is going to be blue&quot; and then be consistent with that. Adversarial nets allow that, by having random inputs. One of the inputs to the NN is a random number, and that random number might determine if the car is going to be blue or red this time.<p>The cool thing about this is that it allows you to generate multiple samples. You can generate 10 different images and select the best one. And the adversarial nets should learn to approximate the true distribution as closely as possible. And I don&#x27;t think there is any other method that can do that.<p>Another idea would to just have a loss function that doesn&#x27;t punish it for getting a wrong color. But rewards it only when it gets very close to the right color. This way the algorithm doesn&#x27;t worry about producing muddy brown colors when it isn&#x27;t&#x27; sure, it just goes with a best guess.