45890100200300Prefix Length (XSUM)18.519.019.520.020.521.0ROUGE-233.534.034.535.035.536.0ROUGE-LROUGE-2ROUGE-L010203040Prefix Length (DART)44.044.545.045.546.0BLEU0.4600.4650.4700.4750.480TERBLEUTERrandom"active""elephant""summarize""table-to-text:""banana""beautiful""divide""keep"0.450.500.550.60BLEUFigure 6: Data efﬁciency curves: percentage of train-
ing set vs. performance on table-to-text (E2E). Preﬁx-
tuning (without the initialization trick) is more data-
efﬁcient than ﬁne-tuning when using more than 20%
of the data.

necessitating the initialization trick (§6.3) to im-
prove the performance in this low-data regime.

8 Discussion
Prefix-Tuning