Stochastic Optimization for Large-scale Optimal Transport
Optimal transport (OT) defines a powerful framework to compare probability distributions in a geometrically faithful way. However, the practical impact of OT is still limited because of its computational burden. We propose a new class of stochastic optimization algorithms to cope with large-scale OT problems. These methods can handle arbitrary distributions (either discrete or continuous) as long as one is able to draw samples from them, which is the typical setup in highdimensional learning problems. This alleviates the need to discretize these densities, while giving access to provably convergent methods that output the correct distance without discretization error. These algorithms rely on two main ideas: (a) the dual OT problem can be re-cast as the maximization of an expectation; (b) the entropic regularization of the primal OT problem yields a smooth dual optimization which can be addressed with algorithms that have a provably faster convergence. We instantiate these ideas in three different setups: (i) when comparing a discrete distribution to another, we show that incremental stochastic optimization schemes can beat Sinkhorn's algorithm, the current state-of-the-art finite dimensional OT solver; (ii) when comparing a discrete distribution to a continuous density, a semidiscrete reformulation of the dual program is amenable to averaged stochastic gradient descent, leading to better performance than approximately solving the problem by discretization ; (iii) when dealing with two continuous densities, we propose a stochastic gradient descent over a reproducing kernel Hilbert space (RKHS). This is currently the only known method to solve this problem, apart from computing OT on finite samples. We backup these claims on a set of discrete, semi-discrete and continuous benchmark problems.
This paper presents methods for the stochastic optimization of large-scale (regularized) optimal transport (OT), which is built upon the dual smoothed representation of entropic regularized OT [Cuturi and Peyré, 2016] (not cited). The paper shows that the dual smoothed objective (appeared in [Cuturi and Peyré, 2016]) can be rewritten differently as a finite separable sum, thus immediately admits stochastic methods by sampling one separate term at a time. As such, the paper explores three different settings of OT, where one or both measures are discrete or continuous, each with one or some illustrative numerical examples. In particular, it proposes to use SAG [16] for the discrete-discrete setting, SGD for the discrete-continuous setting, and RKHS parametrization for the continuous-continuous setting.   In this paper, the convergence complexities are stated inaccurate (see my explanation below), experiments are done very briefly, some important claims are not well justified. The three research problems attempted certainly go beyond an 8-page paper could embrace.   Cuturi, Marco, and Gabriel Peyré. "A smoothed dual approach for variational Wasserstein problems." SIAM Journal on Imaging Sciences 9.1 (2016): 320-343. 
(Update!) The authors have responded to most of my questions in the rebuttal. Given I was the only one who gave a negative score, I would like to summarize my take on their responses.   1. They indeed resolves most of my concerns in the rebuttal, which could have been appeared in their original submission. I believe as long as my questions got cleared out, the paper will meet the threshold of technical quality that NIPS asks for. Particularly, I would like to see:    (a) Clarify that semi-dual formulation is not strongly convex. This message is not in the original paper and can be a plausible excuse that they don't specify the iteration complexity.  (b) Show CPU time comparison between SAG and Sinkhorn per iteration.  (c) Discuss the relation between their RKHS OT approach and the famous MMD kernel approach.  (d) Rephrase their contribution section to avoid inaccurate statements.  (e) Add sensitivity study on ɛ   Given the authors have carefully answered most of my questions, I will raise the technical quality score to 3. Other scores are unchanged.   2. Some views that I made in the review still remains unchanged. But I will let others to decide whether they are valid reasons to reject this paper.   (a) The originality is limited because the semi-dual formulation (new) is only one-step further than the smoothed dual formulation (already existed, but not clearly stated in the paper).  (b) For the D-D case, given there is no theoretical justification that SAG is better than Sinkhorn. The empirical justification is quite brief, not so convincing at this point.  (c) My view on "The authors study three problems in a single 8-page paper. Although they share the same optimal transport formulation, the actual approaches do not really hold any tight connections."   ***** As I summarized, this paper has many issues. Four major problems are   (1) The authors study three problems in a single 8-page paper. Although they share the same optimal transport formulation, the actual approaches do not really hold any tight connections. As a result, this paper leaves many questions unanswered for their approach on each problem, which I’d like to list them one by one later. Some questions I listed are very important, and should be answered to justify a valid algorithmic contribution beyond a half-baked idea.   (2) The formulation (Prop 2.1) are not completely new, Eq. (D_\varepsilon) and later Eq. (\bar{D}_\varepsilon)  already exist in [Cuturi and Peyré, 2016]. Thus, the originality is quite limited.   (3) Some claimed contributions of this paper are not properly justified or ambiguous.    (4) Experiments are not very useful for understanding the convergence properties of each algorithm.   As such, this paper could be improved by focusing on one of the three settings, clarifying and justifying its novel contributions,  and empirically demonstrating its superiority or validity more carefully. More detailed comments are as follows.  (a) Discrete-discrete setting. This is the most practical setting I want to focus in this review.   First of all, there is no evidence in the paper that the SAG approach gives a convergent solution for classical optimal transport. No experiments has done in this regard. Therefore, claims in Line 73 “These methods can be used to solve both classical OT problems and their entropic-regularized versions (which enjoy faster convergence properties). ” are not properly justified.   Second, I want to clarify the differences between SAG and SGD. SAG and many other variance reducing variants are quite different from SGD in terms of their iteration complexity. SAG is linearly convergent (or exponentially convergent ) for smooth, strongly convex problems, whose exponential rate depends on the data size N. SGD has much worse convergence rate, but the iteration complexity does NOT depend on the data size. As such, SAG is more closer to full gradient approach, rather than SGD (lucidly explained in [16]).    SAG, like Sinkhorn iterations, enjoys linear convergence for regularized optimal transport [4,6] due to the use of entropic regularization. In order to justify one is faster than another, one has to either show theoretically and empirically one has a better exponential rate than the other (Please see what has been done for evaluating SAG [16, Table 1], where comparison between SAG and full gradient are made.) Because the iteration complexity of SAG depends on the problem’s conditional number (strongly convex constant and Lipschitz constant), which actually depends on the regularizing parameter ɛ. The comparison of complexities becomes even subtle. Is it possible that SAG becomes slower than Sinkhorn iteration for some ɛ ? Therefore, the statements in Line 78 “while still enjoying a convergence rate O(1/k)” is inaccurate. On one hand, the paper implies SAG has very cheap computation complexity per iteration, which is true, on the other hand, it does not specify the iteration complexity accurately (the dependencies upon N, L and ɛ). In fact, SAG is more expensive in terms of data passes than Sinkhorn.   Third, the paper conducts an experiment comparing SAG and Sinkhorn iterations. Three flaws are (1) as mentioned earlier, sensitivity on ɛ is not explored. ɛ is a crucial parameter for making entropic regularization successful.  (2) empirical exponential rates are not compared.  (3) comparisons are made against passes over data, rather than the actual CPU/GPU time. The iteration of the SAG approach requires the computation of functions exp() and log(), which could be far more expensive than the Sinkhorn iteration. Therefore, the claims in Line 204 “SAG can be more than twice faster than Sinkhorn on average  …” is not proper.   Fourth, as mentioned in [Benamou, 2015] section 1.3, Sinkhorn iteration has a maximal possible iteration number for a small ɛ due to some quantities manipulated in Sinkhorn become smaller than machine precision. Does the SAG approach have this similar issue?  Benamou, Jean-David, et al. "Iterative bregman projections for regularized transportation problems." SIAM Journal on Scientific Computing 37.2 (2015): A1111-A1138.  (b) Discrete-Continuous Setting The paper claims in Line 80 “the SGD approach is numerically advantageous over the brute force approach consisting in sampling first …”. But their empirical validation is rather weak. Because (1) the experimental setting in which a huge number of iterations are used, is advantageous to SGD which converges rather slow. More iterations are clearly not useful for a problem with intrinsic discretization error, but one can increase N to reduce error just like the way SGD asks for more iterations. (2) The ground-truth v^\star “is approximated numerically by running SGD with a large enough number of iterations”. How large is large enough? Please specify the actual number. The logarithmic errors plotted in Figure 2 are known quite sensitive to the small changes of v^\star. As such, it makes the comparison even unfair. (3) Because entropic regularized OT is already an approximation to classical OT, I didn’t see much value to have a very accurate approximation to another approximation.   (c) Continuous-Continuous Setting I didn’t see any practical value or motivation in this approach. It does not compute the exact solution for regularized OT, and converges slowly.   In fact, if u and v are chosen from RKHS rather than some family of continuous functions, the dual OT problem is to compute maximum mean discrepancy, an extensively studied problem in the community of kernel methods. With RKHS embeddings, what’s the motivation to further consider the entropic regularization in regularizing the objective of maximum mean discrepancy? Please see for example [Gretton, 2012].  Gretton, Arthur, et al. "A kernel two-sample test." The Journal of Machine Learning Research 13.1 (2012): 723-773.   I try to give some positive words in the end. I think the SAG approach for Discrete-Discrete setting is an important direction and should be empirically investigated more if there is truly any practical value found in it. The complexity subtleties should be clarified and made easy and accessible to reader, so one can appreciate this idea to some degree. Simply put, for a regularized OT setup (N, L, ɛ), what is the iteration complexity? What is the computation time per iteration? How they compare to Sinkhorn?  Given an OT with m,n support sizes, the current approach mainly considers only one of n,m is large. What if both m and n are large? Is it possible to use doubly stochastic primal-dual approaches, e.g. [Yu, 2015] [Zhang, 2014]? Stochastic optimization of OT is an important direction, and I encourage the authors to improve their work, and re-submit with higher solidility in the future.   Yu, Adams Wei, Qihang Lin, and Tianbao Yang. "Doubly Stochastic Primal-Dual Coordinate Method for Regularized Empirical Risk Minimization with Factorized Data." arXiv preprint arXiv:1508.03390 (2015).  Zhang, Yuchen, and Lin Xiao. "Stochastic primal-dual coordinate method for regularized empirical risk minimization." arXiv preprint arXiv:1409.3257(2014).