Reproducing Bart Xsum from Bart Large
 Summarization: @sshleifer
 Bart: @sshleifer

## Information

I'm trying to finetune bart large for Xsum and unable to reproduce the results from the paper.

When I try eval with facebook/bart-large-xsum, I get R1=45.3595, RLSum=37.1717 so I assume my eval script is working ok. For finetuning bart large, I use the same config as bart-large-xsum with vocab size=50265 to enable starting from bart-large. However, I am unable to reach the same scores. The best I have is R1=45.4188, RLSum=36.6986 with LR=1.2e-4, gbs=128 and --max_target_length=60 --max_source_length=1024 --val_check_interval 0.1 --val_max_target_length=60 --warmup_steps 50 --max_steps 5000. 

How can I reproduce the results?

I don't know the answer to this question. Your numbers are close enough that all I can suggest is to either try fairseq's [command](https://github.com/pytorch/fairseq/blob/master/examples/bart/README.summarization.md#4-fine-tuning-on-cnn-dm-summarization-task) or look at differences between our command and fairseq.
<|||||>> Summarization: @sshleifer Bart: @sshleifer
> 
> ## Information
> I'm trying to finetune bart large for Xsum and unable to reproduce the results from the paper.
> 
> When I try eval with facebook/bart-large-xsum, I get R1=45.3595, RLSum=37.1717 so I assume my eval script is working ok. For finetuning bart large, I use the same config as bart-large-xsum with vocab size=50265 to enable starting from bart-large. However, I am unable to reach the same scores. The best I have is R1=45.4188, RLSum=36.6986 with LR=1.2e-4, gbs=128 and --max_target_length=60 --max_source_length=1024 --val_check_interval 0.1 --val_max_target_length=60 --warmup_steps 50 --max_steps 5000.
> 
> How can I reproduce the results?

Hi @swethmandava . I'm trying to reproduce the result by Transformres. Would you mind sharing your fine-tuning script?