An open secret in contemporary machine learning is that many models work beautifully on standard benchmarks but fail to generalize outside the lab. This has been attributed to biased training data, which provide poor coverage over real world events. Generative models are no exception, but recent advances in generative adversarial networks (GANs) suggest otherwise -these models can now synthesize strikingly realistic and diverse images. Is generative modeling of photos a solved problem? We show that although current GANs can fit standard datasets very well, they still fall short of being comprehensive models of the visual manifold. In particular, we study their ability to fit simple transformations such as camera movements and color changes. We find that the models reflect the biases of the datasets on which they are trained (e.g., centered objects), but that they also exhibit some capacity for generalization: by "steering" in latent space, we can shift the distribution while still creating realistic images. We hypothesize that the degree of distributional shift is related to the breadth of the training data distribution. Thus, we conduct experiments to quantify the limits of GAN transformations and introduce techniques to mitigate the problem.
This paper propose to study the generalization properties of GANs through interpolation. They first propose to learn a linear (and non-linear) interpolation in the latent space for a specific type of image transformation for example zoom, translation, rotation, luminance, etc... They show that linear interpolation in GANs can produce really realistic images along the path and enable to control and transform generated images to some extent. They then propose to measure to what extent the generated images can be transformed without "breaking".  Finally they show that the quality of the interpolation can be improved by learning the interpolation and generator jointly.
I'm in favour of accepting this paper. The paper is well written and organized. The experiments and observations are very interesting and really illustrate the generalization capacity of GANs.
Main argument:
- I think those observations are very valuable to the community and are a good way to get insight into the capabilities of GANs. This also give interesting informations about the different bias present and learnt in the dataset. This could also lead to very nice applications.
- The interpolation with StyleGAN and BigGAN seem to give qualitatively very different results. It would have been very interesting to study the quality of interpolations on more models and datasets, and compare their generalization capabilities as well as the bias present in the different datasets.
- Does training the generator and interpolation jointly improve the quality of the generator in general ? It would have been nice to run this method on more complicated dataset like CIFAR10 and see if this method increase the overall FID score.
Minor comments:
- In appendix A.2 the authors explain how the range of $\alpha$ is set for the different experiments. However it's not clear how is this range used in practice ? Do you sample uniformly $\alpha$ in this range to train the linear interpolation ? Also how many steps are required to learn the linear interpolation ? How much the does it influence the quality of the interpolation ?
- There is a typo in equation 6
- In figure 6: What does the right figure represent ? especially what are the different colours ?