Losslessly Compressed Gemma 3

Dynamic-length Float (DFloat11)
community
AI & ML interests
None defined yet.
Recent Activity
View all activity
Organization Card
⚡️ DFloat11: Lossless LLM Compression for Efficient GPU Inference
DFloat11 is a lossless compression framework that reduces the size of LLMs and Diffusion Models by approximately 30% while preserving bit-for-bit identical outputs to the original model. It enables efficient GPU inference on resource-constrained hardware without sacrificing accuracy.
🚀 Key Features
- Lossless Compression: Achieves ~30% model size reduction with outputs identical to the original BFloat16 models.
- GPU-Efficient: All decompression is handled on-GPU, eliminating CPU overhead and host-device data transfers.
- Scalable Performance: Decompression overhead remains constant per forward pass and is independent of batch size.
- Broad Model Support: Compatible with various models, including Qwen3, Gemma3, Llama3, Phi4, Wan2.1, FLUX.1, and BAGEL.
🛠 Installation
Ensure you have a CUDA-compatible GPU and PyTorch installed.
# For CUDA 12
pip install -U dfloat11[cuda12]
# For CUDA 11
pip install -U dfloat11[cuda11]
🧪 Quick Start
For example usage, refer to the examples directory in the GitHub repository.
📄 Learn More
- Paper: 70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float
- GitHub Repository: LeanModels/DFloat11
- Hugging Face Models: DFloat11 on Hugging Face
Collections
3
models
31

DFloat11/DeepSeek-R1-0528-Qwen3-8B-DF11
Updated
•
7
•
1

DFloat11/BAGEL-7B-MoT-DF11
Any-to-Any
•
Updated
•
1.59k
•
19

DFloat11/Wan2.1-T2V-14B-Diffusers-DF11
Text-to-Image
•
Updated
•
1.46k
•
3

DFloat11/Mistral-Nemo-Instruct-2407-DF11
Updated
•
13

DFloat11/Llama-3.3-70B-Instruct-DF11
Updated
•
17

DFloat11/Mistral-Small-24B-Instruct-2501-DF11
Updated
•
14

DFloat11/stable-diffusion-3.5-large-DF11
Text-to-Image
•
Updated
•
1.3k
•
2

DFloat11/FLUX.1-Fill-dev-DF11
Text-to-Image
•
Updated
•
2.29k
•
2

DFloat11/FLUX.1-Depth-dev-DF11
Text-to-Image
•
Updated
•
728

DFloat11/FLUX.1-Canny-dev-DF11
Text-to-Image
•
Updated
•
1.03k
datasets
0
None public yet