For neural sequence model training, maximum likelihood (ML) has been commonly adopted to optimize model parameters with respect to the corresponding objective. However, in the case of sequence prediction tasks like neural machine translation (NMT), training with the ML-based cross entropy loss would often lead to models that overgeneralize and plunge into local optima. In this paper, we propose an extended loss function called dual skew divergence (DSD), which aims to give a better tradeoff between generalization ability and error avoidance during NMT training. Our empirical study indicates that switching to DSD loss after the convergence of ML training helps the model skip the local optimum and stimulates a stable performance improvement. The evaluations on WMT 2014 English-German and English-French translation tasks demonstrate that the proposed loss indeed helps bring about better translation performance than several baselines.
This paper describes an alternative training objective to cross-entropy loss for sequence-to-sequence models. The key observation is that cross-entropy is minimizing KL(P|Q) for a data distribution P and a model distribution Q; they add another loss that minimizes the inverse KL(Q|P) to create their dual-skew divergence. The idea is tested in the context of neural MT, using a model similar to that proposed by Bahdanau et al. (2015) with results on English-to-French and English-to-German WNT 2014. In the context of beam search, improvements are small (<=0.5 BLEU) but statistically significant.
This is an interesting idea, and one I certainly wouldn’t have thought of on my own, but I think it is currently lacking sufficient experimental support to warrant publication. The paper feels strangely dated, with most experiments on two-layer models, and only two citations from 2017. The experiments compare against an in-house maximum likelihood baseline (varying greedy-vs-beam search and model depth), and against a number of alternative training methods (minimum risk, scheduled sampling, RL) with numbers lifted from various papers. These latter results are not useful, as the authors (helpfully) point out that the baseline results in this paper are universally higher than the baselines from these other papers. Furthermore, it feels like methods designed to address exposure bias and/or BLEU-perplexity mismatch are not the right comparison points for this work, as it does not attempt to address either of these issues. I would instead be much more interested to see a comparison to label smoothing (Szegedy et al., 2015), which perhaps addresses some of the same issues, and which produces roughly the same magnitude of improvements. Also, the literature review should likely be updated to include Edunov et al., 2017. In general, the improvements are small (though technically statistically significant), the baseline models are somewhat shallow and the deltas seem to be decreasing as model depth grows, so it is hard to get too excited.
Smaller concerns:
For Table 1, it would be helpful to explain why Baseline is not equal to \Beta=1. With some effort, I figured out that this was due to the alpha term modifying the cross-entropy objective when \Beta=1.
It would also be useful to tell us what “switching point” was used for Table 1 and Figure 2.
Christian Szegedy, Vincent Vanhoucke, SergeyIoffe, Jonathon Shlens, and Zbigniew Wojna. 2015. Rethinking the inception architecture for computer vision. CoRR abs/1512.00567. http://arxiv.org/abs/1512.00567.
Sergey Edunov, Myle Ott, Michael Auli, David Grangier, and Marc’Aurelio Ranzato. 2018. Classical structured prediction losses for sequence to sequence learning. In Proceedings of NAACL-HLT 2018.