I don’t think you.
Beside MLM objective like BERT-based models, PEGASUS has another special training objective called GSG and that make it powerful for abstractive text summarization.
I want to archive that powerful model for my language