  Recently, sequence-to-sequence (seq2seq) models with the Transformer
architecture have achieved remarkable performance on various conditional text
generation tasks, such as machine translation. However, most of them are
trained with teacher forcing with the ground truth label given at each time
step, without being exposed to incorrectly generated tokens during training,
which hurts its generalization to unseen inputs, that is known as the "exposure
bias" problem. In this work, we propose to mitigate the conditional text
generation problem by contrasting positive pairs with negative pairs, such that
the model is exposed to various valid or incorrect perturbations of the inputs,
for improved generalization. However, training the model with naive contrastive
learning framework using random non-target sequences as negative examples is
suboptimal, since they are easily distinguishable from the correct output,
especially so with models pretrained with large text corpora. Also, generating
positive examples requires domain-specific augmentation heuristics which may
not generalize over diverse domains. To tackle this problem, we propose a
principled method to generate positive and negative samples for contrastive
learning of seq2seq models. Specifically, we generate negative examples by
adding small perturbations to the input sequence to minimize its conditional
likelihood, and positive examples by adding large perturbations while enforcing
it to have a high conditional likelihood. Such "hard" positive and negative
pairs generated using our method guides the model to better distinguish correct
outputs from incorrect ones. We empirically show that our proposed method
significantly improves the generalization of the seq2seq on three text
generation tasks - machine translation, text summarization, and question
generation.

Computation and Language