@sshleifer  I have arxiv
gdown --id 1K2kDBTNXS2ikx9xKmi2Fy0Wsc5u_Lls0 --output arxiv.zip

not pre-processed yet, will process and post a link here. It’s in jsonline format, each line is a json with keys abstract_text, article_text
pinging @anthonyfuller, Anthony is a summrization master, he might have other datasets pre-processed