f -GAN: Training Generative Neural Samplers using Variational Divergence Minimization
Generative neural samplers are probabilistic models that implement sampling using feedforward neural networks: they take a random input vector and produce a sample from a probability distribution defined by the network weights. These models are expressive and allow efficient computation of samples and derivatives, but cannot be used for computing likelihoods or for marginalization. The generativeadversarial training method allows to train such models through the use of an auxiliary discriminative neural network. We show that the generative-adversarial approach is a special case of an existing more general variational divergence estimation approach. We show that any f -divergence can be used for training generative neural samplers. We discuss the benefits of various choices of divergence functions on training complexity and the quality of the obtained generative models.
This paper shows how the GAN framework can be extended to train the model with many different divergences, rather than just the Jensen-Shannon Divergence.
I think this paper should be accepted.  The authors should feel free to take a few sentences from what I say here and put it in the paper: Currently, many researchers working on generative models are interested in the differences between GANs and VAEs. A general consensus, prior to the appearance of this paper on ArXiv, was that VAEs were good at assigning high likelihood to data points, and GANs were good at generating realistic samples, and that this difference is due to the differing cost functions used. Because this paper shows that GANs behave similarly whether they are trained with the KL divergence cost or the cost proposed in [10], that changes the interpretation of the situation. It now seems that GAN samples are high quality due to something about their training procedure or the way the model family is specified, not because of the cost function used to train them. So this paper provides important evidence that is relevant to settling an important scientific question.  It's also possible that the specific divergences provided here might find practical application.   Here are a few things that should be improved, but that I don't think are dealbreakers: - There's one big technical problem here, which is that the bound goes the wrong way. Usually for variational methods, we want to minimize an upper bound on the cost, but here we are minimizing a lower bound. That means that in the general case, the algorithm may exhibit strong non-convergence behavior. This problem is discussed a little bit in this ICLR workshop paper: https://arxiv.org/pdf/1412.6515v4.pdf We also don't really know how much we are improving the true cost, because we don't have any idea how tight the bound is. The algorithm seems to be at least somewhat sane in practice, so it isn't necessarily broken, but it's worth discussing in the paper that the bound doesn't automatically give us all of the theoretical guarantees it gives in the usual variational learning setting. - The same ICLR workshop paper ( https://arxiv.org/pdf/1412.6515v4.pdf ) also shows how to modify GANs to minimize the KL divergence, but in a different way than this paper. It's worth discussing this in the related work. This work shows how to construct a bound on the KL. The workshop paper shows how to make a stochastic approximation to the gradient of the KL that would be unbiased if the discriminator were optimal, but in practice is subject to overfitting and underfitting in the discriminator. The technique in the workshop paper requires departing from the minimax game formulation and instead uses an independent cost for each player. - The original GAN paper [9] was accompanied by an open source code release that included the algorithm presented in sec 3.1, and several follow-up projects based on [9] have used that same procedure, so the algorithm itself is not an original contribution of this paper. However, the proof that it converges is. - Likewise, the practice of monitoring true positive rates / true negative rates is attributed to Larsen and Sønderby, but this was actually done in the original code release accompanying [9]. -  Typo on line 35: "networks" instead of "network" - The review of f-divergences might explain whether there are any desirable properties of the family of f-divergences. This could help motivate the paper a bit more. As the reader, I have the general idea that for an underpowered model, some divergences result in choosing a subset of modes to capture, while others result in averaging modes together, but I don't know exactly what the benefit of having a wide family of divergences is. - Eq 4: I didn't understand why it is necessary to use Jensen's inequality to pull the sup out of the integral. In the case of discrete x, the sup could be pulled outside of the sum with strict equality, as long as the family of functions for T(x) is unrestricted. Is the use of Jensen's inequality just necessary because issues that are specific to continuous x, like restrictions to function families that are integrable / handling cases where the supremum is not actually attained so it's not possible for T(x) to take on a value that results in the supremum? (This is a minor point because I agree that the restriction of the family of T(x) requires an inequality) - In the discussion of Eq 4, it's worth mentioning that in practice, part of the reason we'll have an inequality is that T(x) will be imperfectly optimized. - Numerous issues with Table 2: First, there's one issue with the evaluation itself: The table on the right seems to show that the variational bound reaches nearly its optimal value, but because we don't see (estimates of) the true divergence, we don't have any idea how tight that bound is.  The rest of the issues with table 2 are related to clarity. I spent approximately half an hour reading it, feeling like I was taking an IQ test. These should be easy to fix, given a specification of what confused me:  1) I don't understand what "objective values to the true distribution" means. 2)  I wasn't sure what the F values in the table in the left were. Are these:   a) The theoretical minimum of F   b) The value of F found by your training algorithm   c) The value of F attained when D is at its theoretical minimal?  I think the answer is (a), based on issue #4  3) Are the values in the table on the right the lower bounds or the actual divergence values? I think they are the lower bounds because some of them go lower than the optimal D_f value shown in the table on the left. I would suggest clearly indicating them as bounds by calling them \hat{KL} rather than KL, \hat{JS} rather than JS, etc.  4)  In the table on the right, some values are lower than their minimum value reported on the left. For example, training and testing with KL-rev yields .2414, but in the table on the left its optimal value is reported as .2415. The same thing also happens with JS, with a larger discrepancy. Initially I thought this was an error in the table, but then I realized it is possible I misinterpreted what was being reported for F in the table on the left.   Full disclosure: I've seen this paper on ArXiv, before I was assigned to review it. I didn't know the authors previously, and don't think that knowledge of who wrote it has influenced my ratings.