title: Aya Model: An Instruction Finetuned Open-Access Multilingual Language
    Model
summary: Recent breakthroughs in large language models (LLMs) have centered around a
  handful of data-rich languages. What does it take to broaden access to
  breakthroughs beyond first-class citizen languages? Our work introduces Aya, a
  massively multilingual generative language model that follows instructions in
  101 languages of which over 50% are considered as lower-resourced. Aya
  outperforms mT0 and BLOOMZ on the majority of tasks while covering double the
  number of languages. We introduce extensive new evaluation suites that broaden
  the state-of-art for multilingual eval across 99 languages -- including
  discriminative and generative tasks, human evaluation, and simulated win rates
  that cover both held-out tasks and in-distribution performance. Furthermore, we
  conduct detailed investigations on the optimal finetuning mixture composition,
  data pruning, as well as the toxicity, bias, and safety of our models. We
  open-source our instruction datasets and our model at
  https://hf.co/CohereForAI/aya-101
paper_authors._id: 65cae379d5822659eced5572
paper_authors.name: Ahmet Üstün
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-13T08:24:42.414Z
paper_authors._id: 60d35d7ad7b174177faabd5b
paper_authors.avatarUrl: /avatars/4e5403b9d4a845a2d21e8217dc3c16d2.svg
paper_authors.fullname: Ahmet Üstün
paper_authors.type: user
paper_authors.user: ahmetu
paper_authors._id: 65cae379d5822659eced5573
paper_authors.name: Viraat Aryabumi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-13T08:24:48.751Z
paper_authors._id: 633e9b78acd351fc37d69560
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/633e9b78acd351fc37d69560/rCk8z-C4oOlrksp-QE2lV.jpeg
paper_authors.fullname: Viraat Aryabumi
paper_authors.type: user
paper_authors.user: viraat
paper_authors._id: 65cae379d5822659eced5574
paper_authors.name: Zheng-Xin Yong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-13T08:25:48.057Z
paper_authors._id: 61424bf4f0d914a5f606a823
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61424bf4f0d914a5f606a823/0td8lR4elBaVvJUD9Pojh.png
paper_authors.fullname: Yong Zheng-Xin
paper_authors.type: user
paper_authors.user: yongzx
paper_authors._id: 65cae379d5822659eced5575
paper_authors.name: Wei-Yin Ko
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-13T08:25:54.888Z
paper_authors._id: 640d286c9e5247967ff69840
paper_authors.avatarUrl: /avatars/01b39240236e41859f7a9fd71f83382a.svg
paper_authors.fullname: Wei-Yin Ko
paper_authors.type: user
paper_authors.user: weiyinko
paper_authors._id: 65cae379d5822659eced5576
paper_authors.name: Daniel D'souza
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-13T08:26:14.368Z
paper_authors._id: 657495cbec3bf96e433489a2
paper_authors.avatarUrl: /avatars/3d0f05abaa8c87d37d06521a19d41aee.svg
paper_authors.fullname: Daniel Dsouza
paper_authors.type: user
paper_authors.user: DanielDsouza
paper_authors._id: 65cae379d5822659eced5577
paper_authors.name: Gbemileke Onilude
paper_authors._id: 65cae379d5822659eced5578
paper_authors.name: Neel Bhandari
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-13T08:27:00.364Z
paper_authors._id: 639ece4c0d679f53942c9c23
paper_authors.avatarUrl: /avatars/e253a796a6179fde28323bc3d84ebb90.svg
paper_authors.fullname: Neel Bhandari
paper_authors.type: user
paper_authors.user: neelbhandari
paper_authors._id: 65cae379d5822659eced5579
paper_authors.name: Shivalika Singh
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-13T08:27:07.160Z
paper_authors._id: 62c2175d756039dd0dd20509
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1677753847837-62c2175d756039dd0dd20509.jpeg
paper_authors.fullname: Shivalika Singh
paper_authors.type: user
paper_authors.user: shivalikasingh
paper_authors._id: 65cae379d5822659eced557a
paper_authors.name: Hui-Lee Ooi
paper_authors._id: 65cae379d5822659eced557b
paper_authors.name: Amr Kayid
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-13T08:27:18.115Z
paper_authors._id: 60099e06168be9b5445234b9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/60099e06168be9b5445234b9/l-l9xmtfzhMPAUynFUNsa.png
paper_authors.fullname: Amr Kayid
paper_authors.type: user
paper_authors.user: AmrMKayid
paper_authors._id: 65cae379d5822659eced557c
paper_authors.name: Freddie Vargus
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-13T08:10:10.780Z
paper_authors._id: 618cb927ec9a3fd349fa7e26
paper_authors.avatarUrl: /avatars/75c81a484fcafac3b40ed6ca01832faf.svg
paper_authors.fullname: Freddie Vargus
paper_authors.type: user
paper_authors.user: freddie
paper_authors._id: 65cae379d5822659eced557d
paper_authors.name: Phil Blunsom
paper_authors._id: 65cae379d5822659eced557e
paper_authors.name: Shayne Longpre
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-13T08:27:30.520Z
paper_authors._id: 61f4283a81c4d30f58140242
paper_authors.avatarUrl: /avatars/a1cf1ef1fd442c36ed65c68e51919fed.svg
paper_authors.fullname: Shayne Longpre
paper_authors.type: user
paper_authors.user: Shayne
paper_authors._id: 65cae379d5822659eced557f
paper_authors.name: Niklas Muennighoff
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-13T08:27:37.915Z
paper_authors._id: 5f1eb362eec0ad2a071ad6e2
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/5f1eb362eec0ad2a071ad6e2/IXMYkYKuTwn6kBdWnQeeY.png
paper_authors.fullname: Niklas Muennighoff
paper_authors.type: user
paper_authors.user: Muennighoff
paper_authors._id: 65cae379d5822659eced5580
paper_authors.name: Marzieh Fadaee
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-13T08:27:43.590Z
paper_authors._id: 6441042d5d600fb0951a5f99
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6441042d5d600fb0951a5f99/4CbOaYcEz99BtVAQvnGTn.jpeg
paper_authors.fullname: Marzieh Fadaee
paper_authors.type: user
paper_authors.user: MarziehFadaee
paper_authors._id: 65cae379d5822659eced5581
paper_authors.name: Julia Kreutzer
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-13T08:27:58.197Z
paper_authors._id: 6544e43b12da508864c38f96
paper_authors.avatarUrl: /avatars/76f0cd55b4bf9c03d2686e146c6f795f.svg
paper_authors.fullname: Julia Kreutzer
paper_authors.type: user
paper_authors.user: JuliaKreutzerCohere
paper_authors._id: 65cae379d5822659eced5582
paper_authors.name: Sara Hooker
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-05-27T03:43:44.600Z
paper_authors._id: 63434eb76f59b79da07dbddf
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63434eb76f59b79da07dbddf/BEwmVjqPNYlqmutXG0G6e.jpeg
paper_authors.fullname: Sara Hooker
paper_authors.type: user
paper_authors.user: sarahooker
paper_title: Aya Model: An Instruction Finetuned Open-Access Multilingual Language
    Model
paper_summary: Recent breakthroughs in large language models (LLMs) have centered around a
  handful of data-rich languages. What does it take to broaden access to
  breakthroughs beyond first-class citizen languages? Our work introduces Aya, a
  massively multilingual generative language model that follows instructions in
  101 languages of which over 50% are considered as lower-resourced. Aya
  outperforms mT0 and BLOOMZ on the majority of tasks while covering double the
  number of languages. We introduce extensive new evaluation suites that broaden
  the state-of-art for multilingual eval across 99 languages -- including
  discriminative and generative tasks, human evaluation, and simulated win rates
  that cover both held-out tasks and in-distribution performance. Furthermore, we
  conduct detailed investigations on the optimal finetuning mixture composition,
  data pruning, as well as the toxicity, bias, and safety of our models. We
  open-source our instruction datasets and our model at
  https://hf.co/CohereForAI/aya-101
paper_ai_keywords: massively multilingual generative language model instructions lower-resourced languages mT0 BLOOMZ discriminative tasks generative tasks human evaluation simulated win rates held-out tasks in-distribution performance optimal finetuning mixture composition data pruning toxicity bias safety