BitNet b1.58 2B4T Technical Report
We introduce BitNet b1.58 2B4T, the first open-source, native 1-bit Large
Language Model (LLM) at the 2-billion parameter scale. Trained on a corpus of 4
trillion tokens, the model has been rigorously evaluated across benchmarks
covering language understanding, mathematical reasoning, coding proficiency,
and conversational ability. Our results demonstrate that BitNet b1.58 2B4T
achieves performance on par with leading open-weight, full-precision LLMs of
similar size, while offering significant advantages in computational
efficiency, including substantially reduced memory footprint, energy
consumption, and decoding latency. To facilitate further research and adoption,
the model weights are released via Hugging Face along with open-source
inference implementations for both GPU and CPU architectures.
68006e6e175c8dce4ec17f7a
Shuming Ma
admin_assigned
2025-04-17T08:13:17.676Z
613f07f40153aafa379775f2
/avatars/3965175b320d753d9a5ccb0c7d9298a4.svg
Shuming Ma
user
shumingma
68006e6e175c8dce4ec17f7b
Hongyu Wang
admin_assigned
2025-04-17T08:14:45.597Z
63f71771d36951307fcb4dcd
https://cdn-avatars.huggingface.co/v1/production/uploads/63f71771d36951307fcb4dcd/1c-GSQmSSuDXyVWjxZFy_.jpeg
Hongyu Wang
user
hongyuw
68006e6e175c8dce4ec17f7c
Shaohan Huang
admin_assigned
2025-04-17T08:14:59.750Z
632bd2f72d6a805eeb4bc601
/avatars/6e1533e8a599f3068290aa69ac82cab7.svg
HUANG SHAOHAN
user
buaahsh
68006e6e175c8dce4ec17f7d
Xingxing Zhang
admin_assigned
2025-04-17T08:15:06.359Z
64abbcff6cadc7aca584f71b
/avatars/fc6e85ad4a8befd133a37b411712c648.svg
Xingxing Zhang
user
THU-CHUNXIA
68006e6e175c8dce4ec17f7e
Ying Hu
68006e6e175c8dce4ec17f7f
Ting Song
68006e6e175c8dce4ec17f80
Yan Xia
68006e6e175c8dce4ec17f81
Furu Wei
admin_assigned
2025-04-17T08:15:42.250Z
6368c512fbfe97c16a40baba
/avatars/1c23bc7c0b6d9225699ce27647623d7a.svg
Furu Wei
user
thegenerality
BitNet b1.58 2B4T Technical Report
We introduce BitNet b1.58 2B4T, the first open-source, native 1-bit Large
Language Model (LLM) at the 2-billion parameter scale. Trained on a corpus of 4
trillion tokens, the model has been rigorously evaluated across benchmarks
covering language understanding, mathematical reasoning, coding proficiency,
and conversational ability. Our results demonstrate that BitNet b1.58 2B4T
achieves performance on par with leading open-weight, full-precision LLMs of
similar size, while offering significant advantages in computational
efficiency, including substantially reduced memory footprint, energy
consumption, and decoding latency. To facilitate further research and adoption,
the model weights are released via Hugging Face along with open-source
inference implementations for both GPU and CPU architectures.
BitNet b1.58 2B4T, a 1-bit Large Language Model with 2 billion parameters, matches the performance of full-precision models while improving computational efficiency.
BitNet Large Language Model 1-bit 2-billion parameters corpus token language understanding mathematical reasoning coding proficiency conversational ability computational efficiency memory footprint energy consumption decoding latency Hugging Face inference implementations GPU CPU