For typical sequence prediction problems like language generation, maximum likelihood estimation (MLE) has been commonly adopted as it encourages the predicted sequence most consistent with the ground-truth sequence to have the highest probability of occurring. However, MLE focuses on a once-for-all matching between the predicted sequence and gold-standard consequently, treating all incorrect predictions as being equally incorrect. We call such a drawback negative diversity ignorance in this paper. Treating all incorrect predictions as equal unfairly downplays the nuance of these sequences' detailed token-wise structure. To counteract this, we augment the MLE loss by introducing an extra KL divergence term which is derived from comparing a data-dependent Gaussian prior and the detailed training prediction. The proposed data-dependent Gaussian prior objective (D2GPo) is defined over a prior topological order of tokens, poles apart from the data-independent Gaussian prior (L2 regularization) commonly adopted for smoothing the training of MLE. Experimental results show that the proposed method can effectively make use of more detailed prior in the data and significantly improve the performance of typical language generation tasks, including supervised and unsupervised machine translation, text summarization, storytelling, and image caption.
This paper proposes to add a prior/objective to the standard MLE objective for training text generation models. The prior penalizes incorrect generations/predictions when they are close to the reference; thus, in contrast with standard MLE alone, the training objective does not equally penalize all incorrect predictions. For the experiments, the authors use cosine similarity between fastText embeddings to determine the similarity of a predicted word and the target word. The method is tested on a comprehensive set of text generation tasks: machine translation, unsupervised machine translation, summarization, storytelling, and image captioning. In all cases, simply adding the proposed prior improves over a state-of-the-art model. The results are remarkable, as the proposed prior is useful despite the variety of architectures, tasks (including multi-modal ones), and models with/without pre-training.
In general, it is promising to pursue work in altering the standard MLE objective; changes to learning objective seem orthogonal to the modeling gains made in many papers (as evidenced by the gains the authors show across diverse models). This paper opens up several new directions, i.e., how can we impose even more effective priors? The authors show that it's effective to use a relatively simple fastText-based prior, but it's possible to consider other priors based on large-scale pre-trained language models or learned models. In this vein, a concurrent paper "Neural Text Generation with Unlikelihood Training" has also shown it effective to alter the standard MLE objective. I think it would be nice to discuss this paper and related works. Overall, I think the approach is quite general and elegant.
My main criticism is that the writing was unfocused or unclear at times. The intro discusses a variety of problems in generation, before explaining that the authors only intend to tackle one ("negative diversity ignorance"). It would have been more helpful to read more text in the intro that motivated the problem of negative diversity ignorance and the proposed solution. The second paragraph in the Discussion in Section 4 is rather ambiguous and hand-wavy. It would be nice to see the authors' intuition described more rigorously (i.e., explicitly describing in math how the cosine similarity score is used in the Gaussian prior, or describing in math how the central limit theorem is used). Some of the existing mathematical explanation in section 4 could be made simpler or more clear (the description of f(y) seems to be a distraction since it doesn't end up in the final loss).
I would have also appreciated more analysis. After reading the paper, I have the following questions (which the authors may be able to address in the rebuttal):
* Do off-the-shelf fastText embeddings work well? How important is it to train fastText embeddings on the data itself? If off-the-shelf embeddings worked well, that could make the method easier to use for others in practice.
* How does the gain in performance with D2GPo vary based on the number of training examples? Priors are generally more helpful in low-data regimes. If that is the case here as well, you might get even more compelling results on low-data tasks (all tasks attempted here are somewhat large-scale, as I understand)
* Qualitatively, do you notice any difference in the generations? How does the model make mistakes (are these more "semantic" somehow, i.e. swapping a different synonym in). Perhaps the gaussian prior has some failure modes, i.e., where it increases the probability of very incorrect/opposite words because they have a similar fastText representation. These kinds of intuitions would be useful to know
I also have one technical question:
* When you compare against MASS (Song et al. 2019), do you use the same code and/or pre-trained weights from MASS, or do you pre-train from scratch using the procedure from MASS? (The wording in the text is somewhat ambiguous.) I'm just wondering how comparable the results are vs. MASS, or if it would be useful to know how your version of the pre-trained model does.
Despite my above questions/concerns, I think the proposed method or its predecessors could provide improvements across a variety of text generation tasks, so I overall highly recommend this paper for acceptance.