title: mLongT5: A Multilingual and Efficient Text-To-Text Transformer for
    Longer Sequences
summary: We present our work on developing a multilingual, efficient text-to-text
  transformer that is suitable for handling long inputs. This model, called
  mLongT5, builds upon the architecture of LongT5, while leveraging the
  multilingual datasets used for pretraining mT5 and the pretraining tasks of
  UL2. We evaluate this model on a variety of multilingual summarization and
  question-answering tasks, and the results show stronger performance for mLongT5
  when compared to existing multilingual models such as mBART or M-BERT.
paper_authors._id: 6466fef2cdea48159aa5c354
paper_authors.name: David Uthus
paper_authors._id: 6466fef2cdea48159aa5c355
paper_authors.name: Santiago Ontañón
paper_authors._id: 6466fef2cdea48159aa5c356
paper_authors.name: Joshua Ainslie
paper_authors._id: 6466fef2cdea48159aa5c357
paper_authors.name: Mandy Guo
paper_title: mLongT5: A Multilingual and Efficient Text-To-Text Transformer for
    Longer Sequences
paper_summary: We present our work on developing a multilingual, efficient text-to-text
  transformer that is suitable for handling long inputs. This model, called
  mLongT5, builds upon the architecture of LongT5, while leveraging the
  multilingual datasets used for pretraining mT5 and the pretraining tasks of
  UL2. We evaluate this model on a variety of multilingual summarization and
  question-answering tasks, and the results show stronger performance for mLongT5
  when compared to existing multilingual models such as mBART or M-BERT.
paper_ai_summary: A multilingual text-to-text transformer called mLongT5, combining LongT5 with datasets from mT5 and UL2, outperforms existing models like mBART and M-BERT in summarization and question-answering tasks.
paper_ai_keywords: multilingual text-to-text transformer LongT5 mT5 UL2 summarization question-answering