title: Babel: Open Multilingual Large Language Models Serving Over 90% of
    Global Speakers
summary: Large language models (LLMs) have revolutionized natural language processing
  (NLP), yet open-source multilingual LLMs remain scarce, with existing models
  often limited in language coverage. Such models typically prioritize
  well-resourced languages, while widely spoken but under-resourced languages are
  often overlooked. To address this disparity, we introduce Babel, an
  open multilingual LLM that covers the top 25 languages by number of speakers,
  supports over 90% of the global population, and includes many languages
  neglected by other open multilingual LLMs. Unlike traditional continue
  pretraining approaches, Babel expands its parameter count through a layer
  extension technique that elevates Babel's performance ceiling. We introduce two
  variants: Babel-9B, designed for efficient inference and
  fine-tuning, and Babel-83B, which sets a new standard for open
  multilingual LLMs. Extensive evaluations on multilingual tasks demonstrate its
  superior performance compared to open LLMs of comparable size. In addition,
  using open-source supervised fine-tuning datasets, Babel achieves remarkable
  performance, with Babel-9B-Chat leading among 10B-sized LLMs and Babel-83B-Chat
  setting a new standard for multilingual tasks, reaching the same level of
  commercial models.
paper_authors._id: 67c666245e2443d7d5e9b76a
paper_authors.name: Yiran Zhao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-04T08:51:21.231Z
paper_authors._id: 64802face9ff472e30dc1ceb
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64802face9ff472e30dc1ceb/bcwTlgpaUrU7m2RMB5zCc.png
paper_authors.fullname: Yiran Zhao
paper_authors.type: user
paper_authors.user: Yiran0924
paper_authors._id: 67c666245e2443d7d5e9b76b
paper_authors.name: Chaoqun Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-06T09:27:33.956Z
paper_authors._id: 61657b0b20606e5e73f611cc
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61657b0b20606e5e73f611cc/6ZPne2GYlWkxrx35ND1P8.png
paper_authors.fullname: CHAOQUN LIU
paper_authors.type: user
paper_authors.user: lukecq
paper_authors._id: 67c666245e2443d7d5e9b76c
paper_authors.name: Yue Deng
paper_authors._id: 67c666245e2443d7d5e9b76d
paper_authors.name: Jiahao Ying
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-06T09:35:39.926Z
paper_authors._id: 671609f7664f44a151f1f0e8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/fEQLuH1kdW5Pd9Y_J64hN.png
paper_authors.fullname: jiahao ying
paper_authors.type: user
paper_authors.user: jhying
paper_authors._id: 67c666245e2443d7d5e9b76e
paper_authors.name: Mahani Aljunied
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-06T09:35:33.285Z
paper_authors._id: 6539c87ba318a98bf0d15dd8
paper_authors.avatarUrl: /avatars/beb9ba6eeacb61addc5897836bd59f55.svg
paper_authors.fullname: Mahani Aljunied
paper_authors.type: user
paper_authors.user: maljunied
paper_authors._id: 67c666245e2443d7d5e9b76f
paper_authors.name: Zhaodonghui Li
paper_authors._id: 67c666245e2443d7d5e9b770
paper_authors.name: Lidong Bing
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-06T09:35:19.611Z
paper_authors._id: 6454685a548f22be598414c4
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/eMjMWKJ-AouF7eY1-RzGF.jpeg
paper_authors.fullname: Lidong Bing
paper_authors.type: user
paper_authors.user: LidongBing
paper_authors._id: 67c666245e2443d7d5e9b771
paper_authors.name: Hou Pong Chan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-06T09:35:50.272Z
paper_authors._id: 604f67ef0fe8ff3ec13d71ef
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/604f67ef0fe8ff3ec13d71ef/KhUwWvZ3OJ9nEee3B-SXO.png
paper_authors.fullname: Hou Pong (Ken) Chan
paper_authors.type: user
paper_authors.user: kenchan0226
paper_authors._id: 67c666245e2443d7d5e9b772
paper_authors.name: Yu Rong
paper_authors._id: 67c666245e2443d7d5e9b773
paper_authors.name: Deli Zhao
paper_authors._id: 67c666245e2443d7d5e9b774
paper_authors.name: Wenxuan Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-06T09:27:36.769Z
paper_authors._id: 60dff6ae19a362a8c27862aa
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/60dff6ae19a362a8c27862aa/LIYzLB3cdPh-B3XIBgBCC.jpeg
paper_authors.fullname: Wenxuan Zhang
paper_authors.type: user
paper_authors.user: isakzhang
paper_title: Babel: Open Multilingual Large Language Models Serving Over 90% of
    Global Speakers
paper_summary: Large language models (LLMs) have revolutionized natural language processing
  (NLP), yet open-source multilingual LLMs remain scarce, with existing models
  often limited in language coverage. Such models typically prioritize
  well-resourced languages, while widely spoken but under-resourced languages are
  often overlooked. To address this disparity, we introduce Babel, an
  open multilingual LLM that covers the top 25 languages by number of speakers,
  supports over 90% of the global population, and includes many languages
  neglected by other open multilingual LLMs. Unlike traditional continue
  pretraining approaches, Babel expands its parameter count through a layer
  extension technique that elevates Babel's performance ceiling. We introduce two
  variants: Babel-9B, designed for efficient inference and
  fine-tuning, and Babel-83B, which sets a new standard for open
  multilingual LLMs. Extensive evaluations on multilingual tasks demonstrate its
  superior performance compared to open LLMs of comparable size. In addition,
  using open-source supervised fine-tuning datasets, Babel achieves remarkable
  performance, with Babel-9B-Chat leading among 10B-sized LLMs and Babel-83B-Chat
  setting a new standard for multilingual tasks, reaching the same level of
  commercial models.
paper_ai_keywords: multilingual LLMs natural language processing (NLP) large language models (LLMs) layer extension technique parameter count performance ceiling fine-tuning supervised fine-tuning datasets commercial models