rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking
Paper
•
2501.04519
•
Published
•
195
Davide Cifarelli
davide221
AI & ML interests
codeLLM, reasoning & interpretability
Recent Activity
upvoted
a
paper
4 days ago
rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep
Thinking
liked
a dataset
10 days ago
HuggingFaceH4/MATH-500
updated
a model
16 days ago
KlaraLabs/coconut-qwen-progressive_7
Organizations
davide221's activity
upvoted
a
paper
4 days ago
liked
a
dataset
10 days ago
updated
7
models
16 days ago
KlaraLabs/coconut-qwen-progressive_7
Updated
•
5
KlaraLabs/coconut-qwen-stages_stage_7
Updated
•
6
KlaraLabs/coconut-qwen-stages_stage_6
Updated
•
2
KlaraLabs/coconut-qwen-stages_stage_5
Updated
•
2
KlaraLabs/coconut-qwen-stages_stage_3
Updated
•
2
KlaraLabs/coconut-qwen-stages_stage_2
Updated
•
2
KlaraLabs/coconut-qwen-stages_stage_0
Updated
•
2
updated
a
dataset
17 days ago
liked
a
model
17 days ago
liked
a
model
3 months ago
liked
2
models
4 months ago
liked
a
dataset
4 months ago
liked
a
model
5 months ago
liked
a
model
6 months ago
updated
a
dataset
6 months ago
upvoted
an
article
6 months ago
Article
The Rise of Agentic Data Generation
By
•
•
81
reacted to
yushun0410's
post with 🔥
7 months ago
Post
4624
Hi Huggingfacers!
Thrilled to introduce Adam-mini, an optimizer that achieves on-par or better performance than AdamW with 45% to 50% less memory footprint. Adam-mini can also achieve 49.5% higher throughput than AdamW on Llama2-7B pre-training.
The design of Adam-mini is inspired by certain Hessian structures we observed on Transformers.
Feel free to try it out! Try switching to Adam-mini with the same hyperparams of AdamW, it would work with only half memory. Hope Adam-mini can help save time, cost, and energy in your tasks!
Paper: "Adam-mini: Use Fewer Learning Rates To Gain More" https://arxiv.org/abs/2406.16793
Code: https://github.com/zyushun/Adam-mini
Thrilled to introduce Adam-mini, an optimizer that achieves on-par or better performance than AdamW with 45% to 50% less memory footprint. Adam-mini can also achieve 49.5% higher throughput than AdamW on Llama2-7B pre-training.
The design of Adam-mini is inspired by certain Hessian structures we observed on Transformers.
Feel free to try it out! Try switching to Adam-mini with the same hyperparams of AdamW, it would work with only half memory. Hope Adam-mini can help save time, cost, and energy in your tasks!
Paper: "Adam-mini: Use Fewer Learning Rates To Gain More" https://arxiv.org/abs/2406.16793
Code: https://github.com/zyushun/Adam-mini
- 1 reply