For the summarization task, we use the XSUM
(Narayan et al., 2018) dataset, which is an abstrac-
tive summarization dataset on news articles. There
are 225K examples. The average length of the ar-
ticles is 431 words and the average length of the
summaries is 23.3. We report ROUGE-1, ROUGE-
2 and ROUGE-L, computed by the python package
rouge-score.

Data pre-processing. For table-to-text, we lin-
earize a table x in order to ﬁt into a language model
context. In the E2E dataset, for example, “(ﬁeld A,

12https://github.com/tuetschek/

e2e-metrics

13https://github.com/Yale-LILY/dart

value A), (ﬁeld B, value B)” is linearized to “ﬁeld
A : value A | ﬁeld B : value B”. Also, in WebNLG
and DART, a sequence of triple “(entity1.1, rela-
tion1, entity1.2), (entity2.1, relation2, entity2.2)”
is linearlized as “entity1.1 : relation1 : entity1.2 |
entity2.1 : relation2 : entity2.2”.

For summarization, we truncate the articles x to

512 BPE tokens.
Prefix-Tuning