6 Main Results
6.1 Table-to-text Generation
We ﬁnd that by updating only 0.1% task-speciﬁc pa-
rameters,7 preﬁx-tuning is effective in table-to-text
generation, outperforming other lightweight base-
lines (ADAPTER and FT-TOP2) even by updating
30x fewer parameters and achieving a comparable
performance with (full) ﬁne-tuning. This trend
holds for all datasets: E2E, WebNLG,8 and DART.
If we match the number of parameters for preﬁx-
tuning and adapter-tuning to be 0.1%, Table 2
shows that preﬁx-tuning is signiﬁcantly better than
ADAPTER (0.1%), attaining 4.1 BLEU improve-
ment per dataset on average. Even when we com-
pare with ﬁne-tuning (100%) and adapter-tuning
(3.0%), which update signiﬁcantly more parame-
ters than preﬁx-tuning, preﬁx-tuning still achieves
results comparable or better than those two systems.
This demonstrates that preﬁx-tuning is more Pareto
efﬁcient than adapter-tuning, signiﬁcantly reducing
parameters while improving generation quality.
Prefix-Tuning