IMPROVING SEQUENCE-TO-SEQUENCE LEARNING VIA OPTIMAL TRANSPORT
Sequence-to-sequence models are commonly trained via maximum likelihood estimation (MLE). However, standard MLE training considers a word-level objective, predicting the next word given the previous ground-truth partial sentence. This procedure focuses on modeling local syntactic patterns, and may fail to capture long-range semantic structure. We present a novel solution to alleviate these issues. Our approach imposes global sequence-level guidance via new supervision based on optimal transport, enabling the overall characterization and preservation of semantic features. We further show that this method can be understood as a Wasserstein gradient flow trying to match our model to the ground truth sequence distribution. Extensive experiments are conducted to validate the utility of the proposed approach, showing consistent improvements over a wide variety of NLP tasks, including machine translation, abstractive text summarization, and image captioning.
====== Final Comments =======
I thank the authors for updating the manuscript with clarifications and for clear replies to my concerns. 
I agree with R2 to some extent that the empirical performance of the method, as well as the formulation,  is interesting. In general, the authors addressed my concerns regarding how optimal transport training model interfaces with MLE training and the choice of using scaled softmax for computing the Wasserstein distances. However, I still find myself agreeing with R3 that the choice of sets to compute Wasserstein distances (as opposed to sequences is somewhat unjustified); and it is not clear how the theory in Sec. 3 justifies using sets instead of words, as the data distribution p_d is over sequences in both the W-2 term as well as the MLE term. This would be good to clarify further, or explain more clearly how Sec. 3 justifies this choice.
Also, I missed this in the original review, but the assumption that KL(\mu| p_d) = KL(\p_d| \mu) since \mu is close to p_d for MLE training does not seem like a sensible assumption under model misspecification (as MLE is mode covering). I would suggest the authors discuss this in a revision/ camera-ready version.
In light of these considerations, I am updating the rating to 6 (to reflect the points that have been addressed), but I still do not believe that the method is super-well justified, despite an interesting formulation and strong empirical results (which are aspects the paper could still improve upon). 
=====================
**Summary**
The paper proposes a regularization / fine-tuning scheme in addition to maximum likelihood sequence training using optimal transport. The basic idea is to match a sampled sentence from a model with a ground truth sentence using optimal transport. Experimental results show that the proposed modification improves over MLE training across machine translation, summarization and image captioning domains.
**Strengths**
+ The proposed approach shows promising results across multiple domains.
+ The idea of using optimal transport to match semantics of words intuitively makes sense.
**Weaknesses**
1. Generally, one would think that for optimal transport we would use probabilities which come from the model, i.e. given a set of words and probabilities for each of the words, one would move mass around from one word to another word (present in the ground truth) if the words were semantically relevant to each other and vice versa. However, it seems the proposed algorithm / model does not get the marginal distributions for the words from the model, and indeed does not use any beliefs from the model in the optimal transport formulation / algorithm [Alg. 1, Line 2]. This essentially then means, following objective 2, that optimal transport has no effect on the model’s beliefs on the correct word, but merely needs to ensure that the cosine similarity between words which have a high transport mass is minimized, and has no direct relation to the belief of the model itself. This strikes as a somewhat odd design choice. (*)
2. In general, another issue with the implementation as described in the paper could be the choice of directly using the temperature scaled softmax as an estimate of the argmax from the model, instead of sampling utterances from the model. It seems worth reporting results, even if as a baseline what sampling from something like a concrete distribution [A] yeilds in terms of results. (*)
3. As a confirmation, Is the differentiable formulation for sentences also used for MLE training part of the objective, or does MLE still use ground truth sentences (one-hot encoded) as input. Does this mean that the model is forwarded twice (once with ground truth, and once with softmax outputs)? In general, instead of/ in addition to an Algorithm box that expains the approach of Xie et.al., it would be good to have a clear algorithmic explanation of the training procedure of the proposed model itself. Further, the paper is not clear if the model is always used for finetuning the an MLE model (as Section 3) seems to suggest or if the optimal transport loss is used in conjunction with the model as the bullet ``complementary MLE loss’’ seems to suggest. (*)
4. Section 3: It is not clear that the results / justifications hold for the proposed model since the distance optimized in the current paper is not a wasserstein-2 distance. Sure, it computes cosine distance, which is L-2 but it appears wasserstein-2 distance is defined for continuous probability measures, while the space on which the current paper computes distances is inherently the (discrete) space of word choices. (*)
Experiments
5. The SPICE metric for image captioning takes the content of the sentences (and semantics) into account, instead of checking for fluency. Prior work on using RL techniques for sequence predictions have used SPICE + CIDEr [B] to alleviate the problem with RL methods mentioned in page. 5. Would be good to weaken the claim. (*)
Minor Points
1. It might be good to be more clear about the objective from Xie et.al. by also stating the modified formulation they discuss, and explaining what choice of the function yeilds the bregman divergence of the form discussed in Eqn. 3. 
2. It would be nice to be consistent with how \mathcal{L}_{ot} is used in the paper. Eqn. 4 lists it with two embedding matrices as inputs, while Alg. 1, Line 11 assigns it to be the inner product of two matrices.
**Preliminary Evaluation**
In general, the paper is an interesting take on improving MLE for sequence models. However, while the idea of using optimal transport is interesting and novel for training sequence models, I have questions about the particular way in which it has been implemented, which seems somewhat unjustified. I also have further clarifications about a claimed justification for the model. Given convincing responses for these, and other clarification questions (marked with a *) this would be a good paper.
References
[A]: Maddison, Chris J., Andriy Mnih, and Yee Whye Teh. 2016. “The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables.” arXiv [cs.LG]. arXiv. http://arxiv.org/abs/1611.00712.
[B]: Liu, Siqi, Zhenhai Zhu, Ning Ye, Sergio Guadarrama, and Kevin Murphy. 2016. “Improved Image Captioning via Policy Gradient Optimization of SPIDEr.” arXiv [cs.CV]. arXiv. http://arxiv.org/abs/1612.00370.