We present a novel architecture of GAN for a disentangled representation learning. The new model architecture is inspired by Information Bottleneck (IB) theory thereby named IB-GAN. IB-GAN objective is similar to that of InfoGAN but has a crucial difference; a capacity regularization for mutual information is adopted, thanks to which the generator of IB-GAN can harness a latent representation in disentangled and interpretable manner. To facilitate the optimization of IB-GAN in practice, a new variational upper-bound is derived. With experiments on CelebA, 3DChairs, and dSprites datasets, we demonstrate that the visual quality of samples generated by IB-GAN is often better than those by β-VAEs. Moreover, IB-GAN achieves much higher disentanglement metrics score than β-VAEs or InfoGAN on the dSprites dataset.
(Apologies for this belated review)
Summary 
The authors propose a GAN-based approach to learning disentangled representations that combines elements InfoGAN with recent Information-Bottleneck (IB) perspectives on variational auto-encoders. In addition to minimizing the normal GAN loss, the authors propose to maximize a lower bound on the mutual information under the generative model, whilst minimizing an upper bound
	Ig[X,Z] = E_p(X,Z)[log p(X,Z) - log p(X) - log p(Z)]
In order to optimize this objective whilst retaining the likelihood-free property of GANs, the authors propose to define a generative model with an intermediate representation r, which allows them to define a likelihood-free generator x = G(r) whilst defining a parametric distribution p(r,z) = eψ(r|z) p(z). This enables the authors to define model architectures that jointly train an encoder qφ(z | x) and a GAN-style generator using an objective that incorporates inductive biases for learning disentangled representations
Quantitative evaluation is performed on d-Sprites (where metrics for disentanglement are evaluated), and qualitative results are shown for Celeb-A and the Chairs dataset. 
Comments
I think this is a paper that presents several interesting ideas. Integrating IB-based ideas into the InfoGAN framework is a useful contribution. Moreover, I think that the way they authors integrate a likelihood-free generative model with an inference model is something of a contribution in its own right. I particularly like the idea of the intermediate representation. 
Having done some work in this space, I would say that the results on d-Sprites are quite good. Aside from the numerical scores in the table aside, the latent traversals in Figure 2 show a good degree of disentanglement. There is a reason that many of the recent papers don’t show these traversals; it turns out to quite difficult to disentangle shape from the other variables, and even rotation tends to correlate with some of the other latents in many cases.
That said, I would say that the experiments could do with some additional work. I would like to see some discussion of how tight/loose the upper and lower bounds are (some convergence plots would be helpful in this regard). I would also like to see some experiments that evaluate different choices for λ and β (along with some discussion of how these values were chosen – see below). Finally, could the authors find one or two additional datasets? I generally find it difficult to evaluate results on Celeb-A (other than the qualitative evaluation “the images look sharper than those produced by VAEs”). Even something like MNIST/FMNIST would be OK for purposes of evaluating inclusion of Discrete/Concrete variables and/or extrapolation to unseen combinations of factors (as in the Esmaeli et al. paper). 
Overall, I would say that this is a potentially strong paper, but that experimental evaluation does need work. I’d be willing to look at an updated version of the paper and adjust my score accordingly if the authors can provide one. 
Questions 
- Could the authors comment on why they need to set λ=150, β=1? On a quick read, it is not immediately obvious to me why λ > β implies that we will maximize Ig[X,Z] is this simply because maximizing the lower bound will win out over minimizing the upper bound? When we set λ=β, since this would yield a zero loss when the bounds are tight, is that correct? In this case we presumably not necessarily expect to maximize Ig[X,Z] w.r.t. θ?
- What is perhaps missing from this paper is a discussion of *why* maximizing Ig[X,Z] induces a disentangled representation. One hypothesis could be that be that, for a given number of uncorrelated latent variables, a disentangled generator is simply more efficient in terms of the number of distinct samples X it can construct. In this context, it would be interesting if the the authors could report their Ig[X,Z] bounds. In particular, could they compute
	exp[Ig[X,Z]] / N
 
Intuitively, this number indicates how many examples the generator can produce relative to the number of training examples N. Is it the case that a more disentangled generator is also capable of producing more distinct samples? 
Minor 
- This is a bit of a pet peeve of mine: Is it really true that GANs lean a representation? A representation is generally a mapping from data to features. A GAN is a mapping from features to data. The authors in this paper do train an encoder to invert the generative model, which learns representation, and certainly a disentangled GAN arguably is useful for more controllable forms of generation in its own right, it just seems that we should not conflate the two. 
- Fix: General consent -> General consensus
- Fix: good representation -> good representations
- Fix: such disentangled representation -> disentangled representations
- Fix: (?Higgins et al., 2017b; 2018)