title: Orion-14B: Open-source Multilingual Large Language Models
summary: In this study, we introduce Orion-14B, a collection of multilingual large
  language models with 14 billion parameters. We utilize a data scheduling
  approach to train a foundational model on a diverse corpus of 2.5 trillion
  tokens, sourced from texts in English, Chinese, Japanese, Korean, and other
  languages. Additionally, we fine-tuned a series of models tailored for
  conversational applications and other specific use cases. Our evaluation
  results demonstrate that Orion-14B achieves state-of-the-art performance across
  a broad spectrum of tasks. We make the Orion-14B model family and its
  associated code publicly accessible https://github.com/OrionStarAI/Orion,
  aiming to inspire future research and practical applications in the field.
paper_authors._id: 65b0738c9bd5ea963a85e8de
paper_authors.name: Du Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-24T09:49:26.367Z
paper_authors._id: 64c483deed521f27a4f7dab4
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64c483deed521f27a4f7dab4/J9pKV-3-3ZbsqVPERIIko.png
paper_authors.fullname: chendulearn
paper_authors.type: user
paper_authors.user: chendulearn
paper_authors._id: 65b0738c9bd5ea963a85e8df
paper_authors.name: Yi Huang
paper_authors._id: 65b0738c9bd5ea963a85e8e0
paper_authors.name: Xiaopu Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-16T08:20:19.549Z
paper_authors._id: 61b70ccd7b131aad98e4291c
paper_authors.avatarUrl: /avatars/4c58c9bcedd54d67835cdfa4baba9778.svg
paper_authors.fullname: lee
paper_authors.type: user
paper_authors.user: sharp
paper_authors._id: 65b0738c9bd5ea963a85e8e1
paper_authors.name: Yongqiang Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-24T09:52:13.967Z
paper_authors._id: 63f32e9b0be81bdc5d8f6dd6
paper_authors.avatarUrl: /avatars/1f790160a1199f2e2e0a4b8878a4ca97.svg
paper_authors.fullname: yongqiang li
paper_authors.type: user
paper_authors.user: yqli2420
paper_authors._id: 65b0738c9bd5ea963a85e8e2
paper_authors.name: Yongqiang Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-24T09:51:42.138Z
paper_authors._id: 6540f519c81b3728f0a099db
paper_authors.avatarUrl: /avatars/bd787cee9c5954e6746e4eaaa2151409.svg
paper_authors.fullname: yongqiang liu
paper_authors.type: user
paper_authors.user: liuyongq
paper_authors._id: 65b0738c9bd5ea963a85e8e3
paper_authors.name: Haihui Pan
paper_authors._id: 65b0738c9bd5ea963a85e8e4
paper_authors.name: Leichao Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-24T09:52:39.465Z
paper_authors._id: 643c26d9b409fef15e097937
paper_authors.avatarUrl: /avatars/1fda8d21f1a479db59a30d184c8a8786.svg
paper_authors.fullname: xuleichao
paper_authors.type: user
paper_authors.user: leichao
paper_authors._id: 65b0738c9bd5ea963a85e8e5
paper_authors.name: Dacheng Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-24T09:56:20.210Z
paper_authors._id: 642401213fa01ecba6fd1e9b
paper_authors.avatarUrl: /avatars/c36bce29c1738be9470bcbf084a7c09a.svg
paper_authors.fullname: ZhangDacheng
paper_authors.type: user
paper_authors.user: DachengZhang
paper_authors._id: 65b0738c9bd5ea963a85e8e6
paper_authors.name: Zhipeng Zhang
paper_authors._id: 65b0738c9bd5ea963a85e8e7
paper_authors.name: Kun Han
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-24T09:58:12.384Z
paper_authors._id: 63ec4b3459a46fd5f4b699e3
paper_authors.avatarUrl: /avatars/21d1b4df73b82ffd1b8d31e462daee70.svg
paper_authors.fullname: Kun Han
paper_authors.type: user
paper_authors.user: hankun11
paper_title: Orion-14B: Open-source Multilingual Large Language Models
paper_summary: In this study, we introduce Orion-14B, a collection of multilingual large
  language models with 14 billion parameters. We utilize a data scheduling
  approach to train a foundational model on a diverse corpus of 2.5 trillion
  tokens, sourced from texts in English, Chinese, Japanese, Korean, and other
  languages. Additionally, we fine-tuned a series of models tailored for
  conversational applications and other specific use cases. Our evaluation
  results demonstrate that Orion-14B achieves state-of-the-art performance across
  a broad spectrum of tasks. We make the Orion-14B model family and its
  associated code publicly accessible https://github.com/OrionStarAI/Orion,
  aiming to inspire future research and practical applications in the field.
paper_ai_summary: A multilingual large language model with 14 billion parameters achieves state-of-the-art performance across various tasks through data scheduling and fine-tuning.
paper_ai_keywords: multilingual large language models data scheduling conversational applications state-of-the-art performance