title: Xmodel-2 Technical Report
summary: Xmodel-2 is a 1.2-billion-parameter large language model designed
  specifically for reasoning tasks. Its architecture enables different model
  scales to share a unified set of hyperparameters, allowing for extensive
  experimentation on smaller models and seamless transfer of optimal
  configurations to larger models. To maximize training efficiency and stability,
  Xmodel-2 employs the WSD learning rate scheduler from MiniCPM. Pretrained on
  1.5 trillion tokens from diverse sources, Xmodel-2 achieves state-of-the-art
  performance in complex reasoning and agent-based tasks, while maintaining low
  training costs. These results highlight the potential of efficient model design
  and training strategies in advancing reasoning capabilities. Model checkpoints
  and code are publicly available on GitHub at
  https://github.com/XiaoduoAILab/Xmodel-2
paper_authors._id: 677223b635722632fcc63ff5
paper_authors.name: Wang Qun
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-30T19:29:25.463Z
paper_authors._id: 647adfd0e3a7d24c8e46d7d1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/647adfd0e3a7d24c8e46d7d1/UqZegkj3P2XyPe4Lgnrt7.jpeg
paper_authors.fullname: ValeriaWong
paper_authors.type: user
paper_authors.user: valeriaWong
paper_authors._id: 677223b635722632fcc63ff6
paper_authors.name: Liu Yang
paper_authors._id: 677223b635722632fcc63ff7
paper_authors.name: Lin Qingquan
paper_authors._id: 677223b635722632fcc63ff8
paper_authors.name: Qu Zhijiu
paper_authors._id: 677223b635722632fcc63ff9
paper_authors.name: Jiang Ling
paper_title: Xmodel-2 Technical Report
paper_summary: Xmodel-2 is a 1.2-billion-parameter large language model designed
  specifically for reasoning tasks. Its architecture enables different model
  scales to share a unified set of hyperparameters, allowing for extensive
  experimentation on smaller models and seamless transfer of optimal
  configurations to larger models. To maximize training efficiency and stability,
  Xmodel-2 employs the WSD learning rate scheduler from MiniCPM. Pretrained on
  1.5 trillion tokens from diverse sources, Xmodel-2 achieves state-of-the-art
  performance in complex reasoning and agent-based tasks, while maintaining low
  training costs. These results highlight the potential of efficient model design
  and training strategies in advancing reasoning capabilities. Model checkpoints
  and code are publicly available on GitHub at
  https://github.com/XiaoduoAILab/Xmodel-2
paper_ai_keywords: large language model reasoning tasks hyperparameters WSD learning rate scheduler MiniCPM pretrained tokens state-of-the-art performance complex reasoning agent-based tasks