title: Xmodel-LM Technical Report
summary: We introduce Xmodel-LM, a compact and efficient 1.1B language model
  pre-trained on over 2 trillion tokens. Trained on our self-built dataset
  (Xdata), which balances Chinese and English corpora based on downstream task
  optimization, Xmodel-LM exhibits remarkable performance despite its smaller
  size. It notably surpasses existing open-source language models of similar
  scale. Our model checkpoints and code are publicly accessible on GitHub at
  https://github.com/XiaoduoAILab/XmodelLM.
paper_authors._id: 6661232202a5e5f4374e5492
paper_authors.name: Yichuan Wang
paper_authors._id: 6661232202a5e5f4374e5493
paper_authors.name: Yang Liu
paper_authors._id: 6661232202a5e5f4374e5494
paper_authors.name: Yu Yan
paper_authors._id: 6661232202a5e5f4374e5495
paper_authors.name: Xucheng Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-06T10:18:29.757Z
paper_authors._id: 637dda5b976326508039b2e9
paper_authors.avatarUrl: /avatars/e5bbc245cef4ff5ca3f9e1ab28bb8b87.svg
paper_authors.fullname: Xucheng Huang
paper_authors.type: user
paper_authors.user: hxc123china
paper_authors._id: 6661232202a5e5f4374e5496
paper_authors.name: Ling Jiang
paper_title: Xmodel-LM Technical Report
paper_summary: We introduce Xmodel-LM, a compact and efficient 1.1B language model
  pre-trained on over 2 trillion tokens. Trained on our self-built dataset
  (Xdata), which balances Chinese and English corpora based on downstream task
  optimization, Xmodel-LM exhibits remarkable performance despite its smaller
  size. It notably surpasses existing open-source language models of similar
  scale. Our model checkpoints and code are publicly accessible on GitHub at
  https://github.com/XiaoduoAILab/XmodelLM.
paper_ai_keywords: Xmodel-LM language model pre-trained tokens self-built dataset Xdata corpora down-stream task performance model checkpoints