patrickvonplaten:

Note : The models below were trained without any hyper-parameter search and fp16 precision. For more detail, please refer to the respective model card.


Interesting results!


would love to know how finetune times/inference times compare to bart-base/bart-large.  These are roughly bart-base size, right?


Would also love to know on xsum where gaps between good and worse models get magnified in ROUGE space.


Feels like we desperately need some sort of lb/aggregator, like the one you tried to get going for benchmarking. I know bart-large takes ~24h to get to ~ 21 ROUGE on cnn. @VictorSanh got 15.5 ROUGE2 with bart-base on xsum which felt a little low to me.



Are you using pip install wandb? Share your logs?