InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
This paper describes InfoGAN, an information-theoretic extension to the Generative Adversarial Network that is able to learn disentangled representations in a completely unsupervised manner. InfoGAN is a generative adversarial network that also maximizes the mutual information between a small subset of the latent variables and the observation. We derive a lower bound of the mutual information objective that can be optimized efficiently. Specifically, InfoGAN successfully disentangles writing styles from digit shapes on the MNIST dataset, pose from lighting of 3D rendered images, and background digits from the central digit on the SVHN dataset. It also discovers visual concepts that include hair styles, presence/absence of eyeglasses, and emotions on the CelebA face dataset. Experiments show that InfoGAN learns interpretable representations that are competitive with representations learned by existing supervised methods. For an up-to-date version of this paper, please see https://arxiv.org/abs/1606.03657.
This paper presents an extension of GAN to learn interpretable representations (codes to generate images). The main observation is that the original GAN tends to ignore the additional condition (code) to generate images P(x|z) = P(x|z,c) where c is the code. To address this issue, this paper proposes to maximize the mutual information between the code c and the generated images G(z,c), and derive a variational lower bound for learning with a proposal posterior distribution Q(c|x). The final learning objective is a combination of minmax game loss of GAN and log likelihood of the proposal code distribution. In network implementation, this new code likelihood term is simply added at the end of discriminator with additional linear layers. Experiments are carried out on MNIST, renderings of faces and chairs, SVHN and CelebA with codes being categorical or uniform.
1. ----------------Technical quality: -The descriptions from Line 64 to 77 seem a bit confusing. What I understand is [22] and [23] are weakly-supervised methods while [24] and [7] are actually fully-supervised methods. -The comment in Line 79 seems to be inaccurate. Another notable unsupervised method for disentangling is the work by Cheung et al.: Discovering Hidden Factors of Variation in Deep Networks, ICLR 2015.  -In Line 165, this paper claimed to use factorized Gaussian as code distribution, but it turned out that it used uniform distributions through all the experiments. This should be fixed. -One question about the formulation is about z, the free random vector. I wonder whether z could be considered as the same role as c so that the model only needs to model the mutual information between z and x. If not, what role is z playing in the G(z,c) model? -The proposed algorithm seems to achieve nice results on MNIST: different numbers are well separated in the categorical code. Interesting, the paper actually provides the knowledge about total number of categories for the code distribution (K). This information could be privileged in real-world. I wonder what results can be obtained if the K value is altered.  -This paper presents many qualitative results on five datasets but little quantitative results (only Figure 1 and classification error on MNIST in Line 204). Quantitative evaluation of generated samples has been challenging, but considering the main argument of this paper is to learn disentangled representations for downstream applications, some classification experiments on faces, chairs and house numbers using learned features will be really valuable. 2. ----------------Novelty: +The information theory based derivation of learning disentangled codes is somewhat novel, although it bears similar theoretical background with wake-sleep algorithm and variational auto-encoders (VAEs) regarding joint learning of inference and generation networks. In VAEs, the sampling chain is image -> code -> image while the sampling chain used in this paper is code -> image -> code. Having code in the middle restricts the form of code distribution for enabling end-to-end training. However, in this paper having the code at the two ends of the network grants a flexible use of different code distributions such as categorical or uniform distributions. I believe this is the main advantage of the proposed algorithm. 3. ----------------Impact: +Learning interpretable and disentangled representations with generative models is an important research issue. The maximization of mutual information between code and image provides new insights and improves a popular generative model (GAN). I believe this has certain impact in academics.  4.-----------------Presentation: