Finished Reading
Collection
17 items
•
Updated
Jeffrey Magder
jmagder
AI & ML interests
None yet
Recent Activity
updated
a collection
24 days ago
Finished Reading
upvoted
a
paper
24 days ago
Accuracy is Not All You Need
upvoted
a
paper
about 1 month ago
MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning
Attention
Organizations
None yet
updated
a
collection
24 days ago
upvoted
a
paper
24 days ago
upvoted
a
paper
about 1 month ago
updated
a
collection
about 1 month ago
upvoted
a
collection
about 1 month ago
updated
a
collection
about 2 months ago
upvoted
a
paper
about 2 months ago
updated
a
collection
about 2 months ago
upvoted
a
paper
about 2 months ago
upvoted
an
article
3 months ago
Article
The Large Language Model Course
By
•
•
198
reacted to
danielhanchen's
post with 🔥
3 months ago
Post
6049
🦥 Introducing Unsloth Dynamic v2.0 GGUFs!
Our v2.0 quants set new benchmarks on 5-shot MMLU and KL Divergence, meaning you can now run & fine-tune quantized LLMs while preserving as much accuracy as possible.
Llama 4: unsloth/Llama-4-Scout-17B-16E-Instruct-GGUF
DeepSeek-R1: unsloth/DeepSeek-R1-GGUF-UD
Gemma 3: unsloth/gemma-3-27b-it-GGUF
We made selective layer quantization much smarter. Instead of modifying only a subset of layers, we now dynamically quantize all layers so every layer has a different bit. Now, our dynamic method can be applied to all LLM architectures, not just MoE's.
Blog with Details: https://docs.unsloth.ai/basics/dynamic-v2.0
All our future GGUF uploads will leverage Dynamic 2.0 and our hand curated 300K–1.5M token calibration dataset to improve conversational chat performance.
For accurate benchmarking, we built an evaluation framework to match the reported 5-shot MMLU scores of Llama 4 and Gemma 3. This allowed apples-to-apples comparisons between full-precision vs. Dynamic v2.0, QAT and standard iMatrix quants.
Dynamic v2.0 aims to minimize the performance gap between full-precision models and their quantized counterparts.
Our v2.0 quants set new benchmarks on 5-shot MMLU and KL Divergence, meaning you can now run & fine-tune quantized LLMs while preserving as much accuracy as possible.
Llama 4: unsloth/Llama-4-Scout-17B-16E-Instruct-GGUF
DeepSeek-R1: unsloth/DeepSeek-R1-GGUF-UD
Gemma 3: unsloth/gemma-3-27b-it-GGUF
We made selective layer quantization much smarter. Instead of modifying only a subset of layers, we now dynamically quantize all layers so every layer has a different bit. Now, our dynamic method can be applied to all LLM architectures, not just MoE's.
Blog with Details: https://docs.unsloth.ai/basics/dynamic-v2.0
All our future GGUF uploads will leverage Dynamic 2.0 and our hand curated 300K–1.5M token calibration dataset to improve conversational chat performance.
For accurate benchmarking, we built an evaluation framework to match the reported 5-shot MMLU scores of Llama 4 and Gemma 3. This allowed apples-to-apples comparisons between full-precision vs. Dynamic v2.0, QAT and standard iMatrix quants.
Dynamic v2.0 aims to minimize the performance gap between full-precision models and their quantized counterparts.
liked
a
Space
8 months ago
updated
a
collection
11 months ago
upvoted
a
paper
11 months ago