I just reran xsum and got results close to the paper
python run_eval.py facebook/bart-large-xsum  \
  xsum/test.source \ 
  bart_xsum_test_gens.txt \
  --reference_path xsum/test.target \
  --bs 16 --score_path tmp_rouge.json --fp16

Runtime: 40mins.
{'rouge1': 0.4518
'rouge2': 0.21746,
'rougeL': 0.3636}

(these are fscore.mid_measure values, see calculate_rouge_score)