SCALABLE UNBALANCED OPTIMAL TRANSPORT USING GENERATIVE ADVERSARIAL NETWORKS
Generative adversarial networks (GANs) are an expressive class of neural generative models with tremendous success in modeling high-dimensional continuous measures. In this paper, we present a scalable method for unbalanced optimal transport (OT) based on the generativeadversarial framework. We formulate unbalanced OT as a problem of simultaneously learning a transport map and a scaling factor that push a source measure to a target measure in a cost-optimal manner. We provide theoretical justification for this formulation, showing that it is closely related to an existing static formulation by Liero et al. (2018). We then propose an algorithm for solving this problem based on stochastic alternating gradient updates, similar in practice to GANs, and perform numerical experiments demonstrating how this methodology can be applied to population modeling.
In this paper the authors consider the unbalanced optimal transport problem between two measures with different total mass. The authors introduce first the now standard Kantorovich-like formulation, which considers a coupling whose marginals are penalized to look like the two target measures. The authors introduce a second formulation in (2), somewhat a Kantorovich/Monge hybrid that involves a "random" Monge map where the target point T(x) of a point x now depends also on an additional random variable z, to desribe T(x,z). The authors also consider a local mass creation term (\xi) to weight the initial measure \mu.
The authors emphasize the interest of the 2nd formulation, which, much like the original Monge problem, has an intractable push-forward constraint. This formulation is similar to recent work on Wasserstein Autoencoders (to which is added the scaling parameter). As with WAE, this constraint is relaxed to penalize the deviation between the "random" push-forward and the desired marginal. 
The authors show then that the resulting problem, which involves a transportation cost integrated both on the random variable z and on the input domain x, weighted by xi + a simple penalization for xi + a divergence penalizing the deviation between push-forward and desired marginal, can be optimized altogether by using three NN: 1 for the parameterization of T, 1 for the parameterization of \xi, and one to optimize using a function f a variational bound on the divergence. 2 gradient descents (T,\xi), 1 gradient ascent (f, variational bound).
The authors then make a link between that penalize formulation and something that resembles unbalanced transport (I say resembles because there is some assymetry, and that the type of couplings is restricted). Finally the authors show that by letting increase the penalty in front of the divergence in (6) they recover something that looks like the solution of (2).
For the sake of completeness, the authors provide in the appendix an implementation of a simple dual ascent scheme to approximate unbalanced OT inspired from previous work by Seguy'17, and show that, unlike that work, their implicit parameterization of the scaling factor \xi can help, and illustrate this numerically.
I give credit to the authors for addressing a new problem and providing an algorithmic formulation to do so. That algorithm is itself recovered from an alternative formulation of unbalanced OT, and is therefore interesting in its own right. Unfortunately, I have found the presentation rushed. I really believe the paper would deserve an extensive re-write. Everything is fairly clear until Section 3. Then, the authors introduce their main contribution.  Basically the section tries to prove two things at the same time, without really completing its job. One is to prove that "dualizing" the scaling+ random push-forward equality constraint is ok if one uses big enough regularizers (intuitive), the other that this scaled + random push-forward formulation is closely related to W_{ub}. This is less clear to me (see below). 
The experiments are underwhelming. For faces they happen in latent spaces, and therefore one recovers transport between latent spaces later re-visualized through a decoder. For digits, all is fairly simple. They do not clearly mention whether this alternative UOT approach approximates UOT at all. Despite the title, there's no generation. Therefore my grade is really split between a 5 and a 6.
minor comments and questions:
- Is the reference to a local scaling (\xi) for unbalanced transport entirely new? your paper is not clear on that, and it seems to me this idea already appears in the OT literature.
- I do not understand the connexion you make with GANs. In what sense can you interpret any of your networks as generators? To me it just feels like a simultaneous optimization of various networks, yet without a clear generative purpose. Technically there may be several similarities (as we optimize on networks), but I am not sure this justifies referencing GANs in the title. Additionally, and almost mechanically, putting GAN in your paper, the reader will expect some generation results..
- Numerical benchmarks: Is the technique you propose supposed to approximate the optimal value of Unbalanced OT at all? If yes, is there a way you could compare yourselves with Chizat's approach?
- Somewhere in Lemma 3.2 the fact that you had to use an alternative definition \tilde{W} (by restricting the class of couplings) is not really clarified to the reader. Qualitatively, what does it mean that you restrict the class of couplings to have the same support as \mu? In which situations would \tilde{W} be very different from W_{ub} ? (which, if I understand correctly, only appears in (2) but not elsewhere in the paper?)
- I think it would help for the simple sake of readability to add integration domains under your \int symbols.
- T is used as a subset in Lemma 3.1, while it is used after and before as a map of (x,z)
- T(x,z) looks intuitively like a noisy encoder as in Wasserstein AEs (with, of course, the addition of your term \xi). Could you elaborate?
- I have scanned the paper but did not see how you set lambda.