  Despite being able to generate fluent and grammatical text, current Seq2Seq
summarization models still suffering from the unfaithful generation problem. In
this paper, we study the faithfulness of existing systems from a new
perspective of factual robustness which is the ability to correctly generate
factual information over adversarial unfaithful information. We first measure a
model's factual robustness by its success rate to defend against adversarial
attacks when generating factual information. The factual robustness analysis on
a wide range of current systems shows its good consistency with human judgments
on faithfulness. Inspired by these findings, we propose to improve the
faithfulness of a model by enhancing its factual robustness. Specifically, we
propose a novel training strategy, namely FRSUM, which teaches the model to
defend against both explicit adversarial samples and implicit factual
adversarial perturbations. Extensive automatic and human evaluation results
show that FRSUM consistently improves the faithfulness of various Seq2Seq
models, such as T5, BART.

Computation and Language