Minimum Divergence vs. Maximum Margin: an Empirical Comparison on Seq2Seq Models
Sequence to sequence (seq2seq) models have become a popular framework for neural sequence prediction. While traditional seq2seq models are trained by Maximum Likelihood Estimation (MLE), much recent work has made various attempts to optimize evaluation scores directly to solve the mismatch between training and evaluation, since model predictions are usually evaluated by a task specific evaluation metric like BLEU or ROUGE scores instead of perplexity. This paper puts this existing work into two categories, a) minimum divergence, and b) maximum margin. We introduce a new training criterion based on the analysis of existing work, and empirically compare models in the two categories. Our experimental results show that our new training criterion can usually work better than existing methods, on both the tasks of machine translation and sentence summarization.
 In this paper the authors distinguish between two families of training objectives for seq2seq models, namely, divergence minimization objectives and max-margin objectives. They primarily focus on the divergence minimization family, and show that the MRT and RAML objectives can be related to minimizing the KL divergence between the model*s distribution over outputs and the "exponentiated payoff distribution," with the two objectives differing in terms of the direction of the KL. In addition, the authors propose an objective using the Hellinger distance rather than the KL divergence, and they conduct experiments on machine translation and summarization comparing all the considered objectives. The paper is written extremely clearly, and is a pleasure to read. While the discussion of the relationship between RAML and MRT (and MRT and REINFORCE) is interesting and illuminating, many of these insights appear to have been discussed in earlier papers, and the RAML paper itself notes that it differs from REINFORCE style training in terms of the KL direction. On the other hand, the idea of minimizing Hellinger distance is I believe novel (though related to the alpha-divergence work cited by the authors in the related work section), and it*s nice that training with this loss improves over the other losses. Since the authors* results, however, appear to be somewhat below the state of the art, I think the main question left open by the experimental section is whether training with the Hellinger loss would further improve state of the art models. Even if it would not, it would still be interesting to understand why, and so I think the paper could be strengthened either by outperforming state of the art results or, perhaps through an ablation analysis, showing what aspects of current state of the art models make minimizing the Hellinger loss unnecessary. In summary, Pros: - well written and interesting - a new loss with potential for improvement over other losses - fairly thorough experiments Cons: - much of the analysis is not new - unclear if the proposed loss will improve the state of the art, and if not why Update after author response: thanks for your response. I think the latest revision of the paper is improved, and even though state of the art BLEU scores on IWSLT appear to be in the mid 33s, I think the improvement over the Convolutional Seq2seq model is encouraging, and so I*m increasing my score to 7. I hope you*ll include these newer results in the paper.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Minimum Divergence vs. Maximum Margin: an Empirical Comparison on Seq2Seq Models
Sequence to sequence (seq2seq) models have become a popular framework for neural sequence prediction. While traditional seq2seq models are trained by Maximum Likelihood Estimation (MLE), much recent work has made various attempts to optimize evaluation scores directly to solve the mismatch between training and evaluation, since model predictions are usually evaluated by a task specific evaluation metric like BLEU or ROUGE scores instead of perplexity. This paper puts this existing work into two categories, a) minimum divergence, and b) maximum margin. We introduce a new training criterion based on the analysis of existing work, and empirically compare models in the two categories. Our experimental results show that our new training criterion can usually work better than existing methods, on both the tasks of machine translation and sentence summarization.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 In this paper the authors distinguish between two families of training objectives for seq2seq models, namely, divergence minimization objectives and max-margin objectives. They primarily focus on the divergence minimization family, and show that the MRT and RAML objectives can be related to minimizing the KL divergence between the model*s distribution over outputs and the "exponentiated payoff distribution," with the two objectives differing in terms of the direction of the KL. In addition, the authors propose an objective using the Hellinger distance rather than the KL divergence, and they conduct experiments on machine translation and summarization comparing all the considered objectives. The paper is written extremely clearly, and is a pleasure to read. While the discussion of the relationship between RAML and MRT (and MRT and REINFORCE) is interesting and illuminating, many of these insights appear to have been discussed in earlier papers, and the RAML paper itself notes that it differs from REINFORCE style training in terms of the KL direction. On the other hand, the idea of minimizing Hellinger distance is I believe novel (though related to the alpha-divergence work cited by the authors in the related work section), and it*s nice that training with this loss improves over the other losses. Since the authors* results, however, appear to be somewhat below the state of the art, I think the main question left open by the experimental section is whether training with the Hellinger loss would further improve state of the art models. Even if it would not, it would still be interesting to understand why, and so I think the paper could be strengthened either by outperforming state of the art results or, perhaps through an ablation analysis, showing what aspects of current state of the art models make minimizing the Hellinger loss unnecessary. In summary, Pros: - well written and interesting - a new loss with potential for improvement over other losses - fairly thorough experiments Cons: - much of the analysis is not new - unclear if the proposed loss will improve the state of the art, and if not why Update after author response: thanks for your response. I think the latest revision of the paper is improved, and even though state of the art BLEU scores on IWSLT appear to be in the mid 33s, I think the improvement over the Convolutional Seq2seq model is encouraging, and so I*m increasing my score to 7. I hope you*ll include these newer results in the paper.
assistant
 In this paper the authors distinguish between two families of training objectives for seq2seq models, namely, divergence minimization objectives and max-margin objectives. They primarily focus on the divergence minimization family, and show that the MRT and RAML objectives can be related to minimizing the KL divergence between the model*s distribution over outputs and the "exponentiated payoff distribution," with the two objectives differing in terms of the direction of the KL. In addition, the authors propose an objective using the Hellinger distance rather than the KL divergence, and they conduct experiments on machine translation and summarization comparing all the considered objectives. The paper is written extremely clearly, and is a pleasure to read. While the discussion of the relationship between RAML and MRT (and MRT and REINFORCE) is interesting and illuminating, many of these insights appear to have been discussed in earlier papers, and the RAML paper itself notes that it differs from REINFORCE style training in terms of the KL direction. On the other hand, the idea of minimizing Hellinger distance is I believe novel (though related to the alpha-divergence work cited by the authors in the related work section), and it*s nice that training with this loss improves over the other losses. Since the authors* results, however, appear to be somewhat below the state of the art, I think the main question left open by the experimental section is whether training with the Hellinger loss would further improve state of the art models. Even if it would not, it would still be interesting to understand why, and so I think the paper could be strengthened either by outperforming state of the art results or, perhaps through an ablation analysis, showing what aspects of current state of the art models make minimizing the Hellinger loss unnecessary. In summary, Pros: - well written and interesting - a new loss with potential for improvement over other losses - fairly thorough experiments Cons: - much of the analysis is not new - unclear if the proposed loss will improve the state of the art, and if not why Update after author response: thanks for your response. I think the latest revision of the paper is improved, and even though state of the art BLEU scores on IWSLT appear to be in the mid 33s, I think the improvement over the Convolutional Seq2seq model is encouraging, and so I*m increasing my score to 7. I hope you*ll include these newer results in the paper.