I also want to finetune Pegasus. Thank you for sharing your code! How similar is this to what happens in https://github.com/huggingface/transformers/blob/master/examples/seq2seq/finetune_pegasus_xsum.sh 197 ?