title: BitNet b1.58 2B4T Technical Report
summary: We introduce BitNet b1.58 2B4T, the first open-source, native 1-bit Large
  Language Model (LLM) at the 2-billion parameter scale. Trained on a corpus of 4
  trillion tokens, the model has been rigorously evaluated across benchmarks
  covering language understanding, mathematical reasoning, coding proficiency,
  and conversational ability. Our results demonstrate that BitNet b1.58 2B4T
  achieves performance on par with leading open-weight, full-precision LLMs of
  similar size, while offering significant advantages in computational
  efficiency, including substantially reduced memory footprint, energy
  consumption, and decoding latency. To facilitate further research and adoption,
  the model weights are released via Hugging Face along with open-source
  inference implementations for both GPU and CPU architectures.
paper_authors._id: 68006e6e175c8dce4ec17f7a
paper_authors.name: Shuming Ma
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-17T08:13:17.676Z
paper_authors._id: 613f07f40153aafa379775f2
paper_authors.avatarUrl: /avatars/3965175b320d753d9a5ccb0c7d9298a4.svg
paper_authors.fullname: Shuming Ma
paper_authors.type: user
paper_authors.user: shumingma
paper_authors._id: 68006e6e175c8dce4ec17f7b
paper_authors.name: Hongyu Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-17T08:14:45.597Z
paper_authors._id: 63f71771d36951307fcb4dcd
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63f71771d36951307fcb4dcd/1c-GSQmSSuDXyVWjxZFy_.jpeg
paper_authors.fullname: Hongyu Wang
paper_authors.type: user
paper_authors.user: hongyuw
paper_authors._id: 68006e6e175c8dce4ec17f7c
paper_authors.name: Shaohan Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-17T08:14:59.750Z
paper_authors._id: 632bd2f72d6a805eeb4bc601
paper_authors.avatarUrl: /avatars/6e1533e8a599f3068290aa69ac82cab7.svg
paper_authors.fullname: HUANG SHAOHAN
paper_authors.type: user
paper_authors.user: buaahsh
paper_authors._id: 68006e6e175c8dce4ec17f7d
paper_authors.name: Xingxing Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-17T08:15:06.359Z
paper_authors._id: 64abbcff6cadc7aca584f71b
paper_authors.avatarUrl: /avatars/fc6e85ad4a8befd133a37b411712c648.svg
paper_authors.fullname: Xingxing Zhang
paper_authors.type: user
paper_authors.user: THU-CHUNXIA
paper_authors._id: 68006e6e175c8dce4ec17f7e
paper_authors.name: Ying Hu
paper_authors._id: 68006e6e175c8dce4ec17f7f
paper_authors.name: Ting Song
paper_authors._id: 68006e6e175c8dce4ec17f80
paper_authors.name: Yan Xia
paper_authors._id: 68006e6e175c8dce4ec17f81
paper_authors.name: Furu Wei
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-17T08:15:42.250Z
paper_authors._id: 6368c512fbfe97c16a40baba
paper_authors.avatarUrl: /avatars/1c23bc7c0b6d9225699ce27647623d7a.svg
paper_authors.fullname: Furu Wei
paper_authors.type: user
paper_authors.user: thegenerality
paper_title: BitNet b1.58 2B4T Technical Report
paper_summary: We introduce BitNet b1.58 2B4T, the first open-source, native 1-bit Large
  Language Model (LLM) at the 2-billion parameter scale. Trained on a corpus of 4
  trillion tokens, the model has been rigorously evaluated across benchmarks
  covering language understanding, mathematical reasoning, coding proficiency,
  and conversational ability. Our results demonstrate that BitNet b1.58 2B4T
  achieves performance on par with leading open-weight, full-precision LLMs of
  similar size, while offering significant advantages in computational
  efficiency, including substantially reduced memory footprint, energy
  consumption, and decoding latency. To facilitate further research and adoption,
  the model weights are released via Hugging Face along with open-source
  inference implementations for both GPU and CPU architectures.
paper_ai_summary: BitNet b1.58 2B4T, a 1-bit Large Language Model with 2 billion parameters, matches the performance of full-precision models while improving computational efficiency.
paper_ai_keywords: BitNet Large Language Model 1-bit 2-billion parameters corpus token language understanding mathematical reasoning coding proficiency conversational ability computational efficiency memory footprint energy consumption decoding latency Hugging Face inference implementations GPU CPU