Large Scale Optimal Transport and Mapping Estimation
This paper presents a novel two-step approach for the fundamental problem of learning an optimal map from one distribution to another. First, we learn an optimal transport (OT) plan, which can be thought as a one-to-many map between the two distributions. To that end, we propose a stochastic dual approach of regularized OT, and show empirically that it scales better than a recent related approach when the amount of samples is very large. Second, we estimate a Monge map as a deep neural network learned by approximating the barycentric projection of the previously-obtained OT plan. This parameterization allows generalization of the mapping outside the support of the input measure. We prove two theoretical stability results of regularized OT which show that our estimations converge to the OT and Monge map between the underlying continuous measures. We showcase our proposed approach on two applications: domain adaptation and generative modeling.
 Quality The theoretical results presented in the paper appear to be correct. However, the experimental evaluation is globally limited, hyperparameter tuning on test which is not fair. Clarity The paper is mostly clear, even though some parts deserve more discussion/clarification (algorithm, experimental evaluation). Originality The theoretical results are original, and the SGD approach is a priori original as well. Significance The relaxed dual formulation and OT/Monge maps convergence results are interesting and can of of interest for researchers in the area, the other aspects of the paper are limited. Pros: -Theoretical results on the convergence of OT/Monge maps -Regularized formulation compatible with SGD Cons -Experimental evaluation limited -The large scale aspect lacks of thorough analysis -The paper presents 2 contributions but at then end of the day, the development of each of them appears limited Comments: -The weak convergence results are interesting. However, the fact that no convergence rate is given makes the result weak. In particular, it is possible that the number of examples needed for achieving a given approximation is at least exponential. This can be coherent with the problem of Domain Adaptation that can be NP-hard even under the co-variate shift assumption (Ben-David&Urner, ALT2012). Then, I think that the claim of page 6 saying that Domain Adaptation can be performed "nearly optimally" has then to be rephrased. I think that results show that the approach is theoretically justified but optimality is not here yet. Theorem 1 is only valid for entropy-based regularizations, what is the difficulty for having a similar result with L2 regularization? -The experimental evaluation on the running time is limited to one particular problem. If this subject is important, it would have been interesting to compare the approaches on other large scale problems and possibly with other implementations. It is also surprising that the efficiency the L2-regularized version is not evaluated. For a paper interesting in large scale aspects, the experimental evaluation is rather weak. The 2 methods compared in Fig 2 reach the same objective values at convergence, but is there any particular difference in the solutions found? -Algorithm 1 is presented without any discussion about complexity, rate of convergence. Could the authors discuss this aspect? The presentation of this algo is a bit short and could deserve more space (in the supplementary) -For the DA application, the considered datasets are classic but not really "large scale", anyway this is a minor remark. The setup is not completely clear, since the approach is interesting for out of sample data, so I would expect the map to be computed on a small sample of source data, and then all source instances to be projected on target with the learned map. This point is not very clear and we do not know how many source instances are used to compute the mapping - the mapping is incomplete on this point while this is an interesting aspect of the paper: this justifies even more the large scale aspect is the algo need less examples during learning to perform similar or even better classification. Hyperparameter tuning is another aspect that is not sufficiently precise in the experimental setup: it seems that the parameters are tuned on test (for all methods), which is not fair since target label information will not be available from a practical standpoint. The authors claim that they did not want to compete with state of the art DA, but the approach of Perrot et al., 2016 seems to a have a similar objective and could be used as a baseline. Experiments on generative optimal transport are interesting and probably generate more discussion/perspectives. -- After rebuttal -- Authors have answered to many of my comments, I think this is an interesting paper, I increase my score.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Large Scale Optimal Transport and Mapping Estimation
This paper presents a novel two-step approach for the fundamental problem of learning an optimal map from one distribution to another. First, we learn an optimal transport (OT) plan, which can be thought as a one-to-many map between the two distributions. To that end, we propose a stochastic dual approach of regularized OT, and show empirically that it scales better than a recent related approach when the amount of samples is very large. Second, we estimate a Monge map as a deep neural network learned by approximating the barycentric projection of the previously-obtained OT plan. This parameterization allows generalization of the mapping outside the support of the input measure. We prove two theoretical stability results of regularized OT which show that our estimations converge to the OT and Monge map between the underlying continuous measures. We showcase our proposed approach on two applications: domain adaptation and generative modeling.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 Quality The theoretical results presented in the paper appear to be correct. However, the experimental evaluation is globally limited, hyperparameter tuning on test which is not fair. Clarity The paper is mostly clear, even though some parts deserve more discussion/clarification (algorithm, experimental evaluation). Originality The theoretical results are original, and the SGD approach is a priori original as well. Significance The relaxed dual formulation and OT/Monge maps convergence results are interesting and can of of interest for researchers in the area, the other aspects of the paper are limited. Pros: -Theoretical results on the convergence of OT/Monge maps -Regularized formulation compatible with SGD Cons -Experimental evaluation limited -The large scale aspect lacks of thorough analysis -The paper presents 2 contributions but at then end of the day, the development of each of them appears limited Comments: -The weak convergence results are interesting. However, the fact that no convergence rate is given makes the result weak. In particular, it is possible that the number of examples needed for achieving a given approximation is at least exponential. This can be coherent with the problem of Domain Adaptation that can be NP-hard even under the co-variate shift assumption (Ben-David&Urner, ALT2012). Then, I think that the claim of page 6 saying that Domain Adaptation can be performed "nearly optimally" has then to be rephrased. I think that results show that the approach is theoretically justified but optimality is not here yet. Theorem 1 is only valid for entropy-based regularizations, what is the difficulty for having a similar result with L2 regularization? -The experimental evaluation on the running time is limited to one particular problem. If this subject is important, it would have been interesting to compare the approaches on other large scale problems and possibly with other implementations. It is also surprising that the efficiency the L2-regularized version is not evaluated. For a paper interesting in large scale aspects, the experimental evaluation is rather weak. The 2 methods compared in Fig 2 reach the same objective values at convergence, but is there any particular difference in the solutions found? -Algorithm 1 is presented without any discussion about complexity, rate of convergence. Could the authors discuss this aspect? The presentation of this algo is a bit short and could deserve more space (in the supplementary) -For the DA application, the considered datasets are classic but not really "large scale", anyway this is a minor remark. The setup is not completely clear, since the approach is interesting for out of sample data, so I would expect the map to be computed on a small sample of source data, and then all source instances to be projected on target with the learned map. This point is not very clear and we do not know how many source instances are used to compute the mapping - the mapping is incomplete on this point while this is an interesting aspect of the paper: this justifies even more the large scale aspect is the algo need less examples during learning to perform similar or even better classification. Hyperparameter tuning is another aspect that is not sufficiently precise in the experimental setup: it seems that the parameters are tuned on test (for all methods), which is not fair since target label information will not be available from a practical standpoint. The authors claim that they did not want to compete with state of the art DA, but the approach of Perrot et al., 2016 seems to a have a similar objective and could be used as a baseline. Experiments on generative optimal transport are interesting and probably generate more discussion/perspectives. -- After rebuttal -- Authors have answered to many of my comments, I think this is an interesting paper, I increase my score.
assistant
 Quality The theoretical results presented in the paper appear to be correct. However, the experimental evaluation is globally limited, hyperparameter tuning on test which is not fair. Clarity The paper is mostly clear, even though some parts deserve more discussion/clarification (algorithm, experimental evaluation). Originality The theoretical results are original, and the SGD approach is a priori original as well. Significance The relaxed dual formulation and OT/Monge maps convergence results are interesting and can of of interest for researchers in the area, the other aspects of the paper are limited. Pros: -Theoretical results on the convergence of OT/Monge maps -Regularized formulation compatible with SGD Cons -Experimental evaluation limited -The large scale aspect lacks of thorough analysis -The paper presents 2 contributions but at then end of the day, the development of each of them appears limited Comments: -The weak convergence results are interesting. However, the fact that no convergence rate is given makes the result weak. In particular, it is possible that the number of examples needed for achieving a given approximation is at least exponential. This can be coherent with the problem of Domain Adaptation that can be NP-hard even under the co-variate shift assumption (Ben-David&Urner, ALT2012). Then, I think that the claim of page 6 saying that Domain Adaptation can be performed "nearly optimally" has then to be rephrased. I think that results show that the approach is theoretically justified but optimality is not here yet. Theorem 1 is only valid for entropy-based regularizations, what is the difficulty for having a similar result with L2 regularization? -The experimental evaluation on the running time is limited to one particular problem. If this subject is important, it would have been interesting to compare the approaches on other large scale problems and possibly with other implementations. It is also surprising that the efficiency the L2-regularized version is not evaluated. For a paper interesting in large scale aspects, the experimental evaluation is rather weak. The 2 methods compared in Fig 2 reach the same objective values at convergence, but is there any particular difference in the solutions found? -Algorithm 1 is presented without any discussion about complexity, rate of convergence. Could the authors discuss this aspect? The presentation of this algo is a bit short and could deserve more space (in the supplementary) -For the DA application, the considered datasets are classic but not really "large scale", anyway this is a minor remark. The setup is not completely clear, since the approach is interesting for out of sample data, so I would expect the map to be computed on a small sample of source data, and then all source instances to be projected on target with the learned map. This point is not very clear and we do not know how many source instances are used to compute the mapping - the mapping is incomplete on this point while this is an interesting aspect of the paper: this justifies even more the large scale aspect is the algo need less examples during learning to perform similar or even better classification. Hyperparameter tuning is another aspect that is not sufficiently precise in the experimental setup: it seems that the parameters are tuned on test (for all methods), which is not fair since target label information will not be available from a practical standpoint. The authors claim that they did not want to compete with state of the art DA, but the approach of Perrot et al., 2016 seems to a have a similar objective and could be used as a baseline. Experiments on generative optimal transport are interesting and probably generate more discussion/perspectives. -- After rebuttal -- Authors have answered to many of my comments, I think this is an interesting paper, I increase my score.