title: Aya 23: Open Weight Releases to Further Multilingual Progress
summary: This technical report introduces Aya 23, a family of multilingual language
  models. Aya 23 builds on the recent release of the Aya model (\"Ust\"un et al.,
  2024), focusing on pairing a highly performant pre-trained model with the
  recently released Aya collection (Singh et al., 2024). The result is a powerful
  multilingual large language model serving 23 languages, expanding state-of-art
  language modeling capabilities to approximately half of the world's population.
  The Aya model covered 101 languages whereas Aya 23 is an experiment in depth vs
  breadth, exploring the impact of allocating more capacity to fewer languages
  that are included during pre-training. Aya 23 outperforms both previous
  massively multilingual models like Aya 101 for the languages it covers, as well
  as widely used models like Gemma, Mistral and Mixtral on an extensive range of
  discriminative and generative tasks. We release the open weights for both the
  8B and 35B models as part of our continued commitment for expanding access to
  multilingual progress.
paper_authors._id: 6653ed0583fac8ccdc09c6fd
paper_authors.name: Viraat Aryabumi
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-05-27T04:13:50.260Z
paper_authors._id: 633e9b78acd351fc37d69560
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/633e9b78acd351fc37d69560/rCk8z-C4oOlrksp-QE2lV.jpeg
paper_authors.fullname: Viraat Aryabumi
paper_authors.type: user
paper_authors.user: viraat
paper_authors._id: 6653ed0583fac8ccdc09c6fe
paper_authors.name: John Dang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-27T08:49:56.573Z
paper_authors._id: 65c581dfc3fa039f843991f6
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65c581dfc3fa039f843991f6/O4ywGgQQrzps5JYeR3o0y.jpeg
paper_authors.fullname: John Dang
paper_authors.type: user
paper_authors.user: johndang-cohere
paper_authors._id: 6653ed0583fac8ccdc09c6ff
paper_authors.name: Dwarak Talupuru
paper_authors._id: 6653ed0583fac8ccdc09c700
paper_authors.name: Saurabh Dash
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-27T08:50:22.027Z
paper_authors._id: 63adeb79a31e9ea8298ad4ac
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63adeb79a31e9ea8298ad4ac/a_kEJXPLkfaHDUN3XcTYA.jpeg
paper_authors.fullname: Saurabh Dash
paper_authors.type: user
paper_authors.user: saurabhdash
paper_authors._id: 6653ed0583fac8ccdc09c701
paper_authors.name: David Cairuz
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-27T08:52:57.723Z
paper_authors._id: 64674f1a8334813a7ae33023
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64674f1a8334813a7ae33023/VjxQYzaXW7zlH7ktPBsEX.jpeg
paper_authors.fullname: David Cairuz
paper_authors.type: user
paper_authors.user: davidcairuz
paper_authors._id: 6653ed0583fac8ccdc09c702
paper_authors.name: Hangyu Lin
paper_authors._id: 6653ed0583fac8ccdc09c703
paper_authors.name: Bharat Venkitesh
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-27T08:53:51.118Z
paper_authors._id: 65e8e96868cd34406483a639
paper_authors.avatarUrl: /avatars/77fbe9ebd1fccd2e15abb403e2139652.svg
paper_authors.fullname: Bharat venkitesh
paper_authors.type: user
paper_authors.user: bharatv007
paper_authors._id: 6653ed0583fac8ccdc09c704
paper_authors.name: Madeline Smith
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-27T08:53:58.583Z
paper_authors._id: 6351689040dffad651f0d26b
paper_authors.avatarUrl: /avatars/65808f9bf9243c7e8bf7130cdf86166b.svg
paper_authors.fullname: Madeline Smith
paper_authors.type: user
paper_authors.user: madelzzz
paper_authors._id: 6653ed0583fac8ccdc09c705
paper_authors.name: Kelly Marchisio
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-27T08:54:17.273Z
paper_authors._id: 63bc6313d8d676a229a2b9c8
paper_authors.avatarUrl: /avatars/d11fa232a03d5d0ac88c177ef1cf187d.svg
paper_authors.fullname: Kelly Marchisio
paper_authors.type: user
paper_authors.user: kellymarchisio
paper_authors._id: 6653ed0583fac8ccdc09c706
paper_authors.name: Sebastian Ruder
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-27T08:54:26.270Z
paper_authors._id: 624575789dc26c59bae6a19e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1648719221694-noauth.jpeg
paper_authors.fullname: Sebastian Ruder
paper_authors.type: user
paper_authors.user: ruder
paper_authors._id: 6653ed0583fac8ccdc09c707
paper_authors.name: Acyr Locatelli
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-27T08:54:35.114Z
paper_authors._id: 60d370b613f774189902f556
paper_authors.avatarUrl: /avatars/717bd7955f30eb76310d2074fd51afce.svg
paper_authors.fullname: Acyr Locatelli
paper_authors.type: user
paper_authors.user: acyrl
paper_authors._id: 6653ed0583fac8ccdc09c708
paper_authors.name: Julia Kreutzer
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-27T08:54:49.294Z
paper_authors._id: 6544e43b12da508864c38f96
paper_authors.avatarUrl: /avatars/76f0cd55b4bf9c03d2686e146c6f795f.svg
paper_authors.fullname: Julia Kreutzer
paper_authors.type: user
paper_authors.user: JuliaKreutzerCohere
paper_authors._id: 6653ed0583fac8ccdc09c709
paper_authors.name: Nick Frosst
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-27T08:54:57.514Z
paper_authors._id: 6402473006c715b934011e0d
paper_authors.avatarUrl: /avatars/e312516c1e0edbf65418b5285b3164d4.svg
paper_authors.fullname: nick frosst
paper_authors.type: user
paper_authors.user: nickfrosst
paper_authors._id: 6653ed0583fac8ccdc09c70a
paper_authors.name: Phil Blunsom
paper_authors._id: 6653ed0583fac8ccdc09c70b
paper_authors.name: Marzieh Fadaee
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-27T08:55:08.278Z
paper_authors._id: 6441042d5d600fb0951a5f99
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6441042d5d600fb0951a5f99/4CbOaYcEz99BtVAQvnGTn.jpeg
paper_authors.fullname: Marzieh Fadaee
paper_authors.type: user
paper_authors.user: MarziehFadaee
paper_authors._id: 6653ed0583fac8ccdc09c70c
paper_authors.name: Ahmet Üstün
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-27T08:55:16.611Z
paper_authors._id: 60d35d7ad7b174177faabd5b
paper_authors.avatarUrl: /avatars/4e5403b9d4a845a2d21e8217dc3c16d2.svg
paper_authors.fullname: Ahmet Üstün
paper_authors.type: user
paper_authors.user: ahmetu
paper_authors._id: 6653ed0583fac8ccdc09c70d
paper_authors.name: Sara Hooker
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-05-27T03:43:46.321Z
paper_authors._id: 63434eb76f59b79da07dbddf
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63434eb76f59b79da07dbddf/BEwmVjqPNYlqmutXG0G6e.jpeg
paper_authors.fullname: Sara Hooker
paper_authors.type: user
paper_authors.user: sarahooker
paper_title: Aya 23: Open Weight Releases to Further Multilingual Progress
paper_summary: This technical report introduces Aya 23, a family of multilingual language
  models. Aya 23 builds on the recent release of the Aya model (\"Ust\"un et al.,
  2024), focusing on pairing a highly performant pre-trained model with the
  recently released Aya collection (Singh et al., 2024). The result is a powerful
  multilingual large language model serving 23 languages, expanding state-of-art
  language modeling capabilities to approximately half of the world's population.
  The Aya model covered 101 languages whereas Aya 23 is an experiment in depth vs
  breadth, exploring the impact of allocating more capacity to fewer languages
  that are included during pre-training. Aya 23 outperforms both previous
  massively multilingual models like Aya 101 for the languages it covers, as well
  as widely used models like Gemma, Mistral and Mixtral on an extensive range of
  discriminative and generative tasks. We release the open weights for both the
  8B and 35B models as part of our continued commitment for expanding access to
  multilingual progress.
paper_ai_keywords: multilingual language models end-performant pre-trained model Aya collection state-of-art language modeling pre-training discriminative tasks generative tasks