For summarization, we truncate the articles x to

512 BPE tokens.

Extrapolation data splits. We construct two ex-
trapolation data splits news-to-sports and
within-news from the original XSUM dataset.
XSUM dataset is drawn from BBC news, and we
identify the topic of each article based on its URL.
Since “news” and “sports” are the two domains
with the most articles, we create our ﬁrst train/test
split. Additionally, “news” has subdomains such as
“UK”, “world”, and “technology”. Consequently,
we create a second data split, using the top 3 news
subdomains (i.e. {world, UK, business }) as train-
ing data and the rest as test data.
Prefix-Tuning