SmolTulu: Higher Learning Rate to Batch Size Ratios Can Lead to Better
  Reasoning in SLMs
We present SmolTulu-1.7b-Instruct, referenced in this report as
SmolTulu-DPO-1130, an instruction-tuned language model that adapts AllenAI's
Tulu 3 post-training pipeline to enhance Huggingface's SmolLM2-1.7B base model.
Through comprehensive empirical analysis using a 135M parameter model, we
demonstrate that the relationship between learning rate and batch size
significantly impacts model performance in a task-dependent manner. Our
findings reveal a clear split: reasoning tasks like ARC and GSM8K benefit from
higher learning rate to batch size ratios, while pattern recognition tasks such
as HellaSwag and IFEval show optimal performance with lower ratios. These
insights informed the development of SmolTulu, which achieves state-of-the-art
performance among sub-2B parameter models on instruction following, scoring
67.7% on IFEval (Delta11%), and mathematical reasoning with 51.6% on GSM8K
(Delta3.4%), with an alternate version achieving scoring 57.1% on ARC
(Delta5.4%). We release our model, training recipes, and ablation studies to
facilitate further research in efficient model alignment, demonstrating that
careful adaptation of optimization dynamics can help bridge the capability gap
between small and large language models.
675ac92733f6ac95ba7d6336
Sultan Alrashed
claimed_verified
2024-12-12T13:41:00.392Z
63fc685a4c57549ad5e5c419
https://cdn-avatars.huggingface.co/v1/production/uploads/63fc685a4c57549ad5e5c419/0oySJfCD7BUVD59kXyhsV.jpeg
Sultan Alrashed
user
SultanR
SmolTulu: Higher Learning Rate to Batch Size Ratios Can Lead to Better
  Reasoning in SLMs
We present SmolTulu-1.7b-Instruct, referenced in this report as
SmolTulu-DPO-1130, an instruction-tuned language model that adapts AllenAI's
Tulu 3 post-training pipeline to enhance Huggingface's SmolLM2-1.7B base model.
Through comprehensive empirical analysis using a 135M parameter model, we
demonstrate that the relationship between learning rate and batch size
significantly impacts model performance in a task-dependent manner. Our
findings reveal a clear split: reasoning tasks like ARC and GSM8K benefit from
higher learning rate to batch size ratios, while pattern recognition tasks such
as HellaSwag and IFEval show optimal performance with lower ratios. These
insights informed the development of SmolTulu, which achieves state-of-the-art
performance among sub-2B parameter models on instruction following, scoring
67.7% on IFEval (Delta11%), and mathematical reasoning with 51.6% on GSM8K
(Delta3.4%), with an alternate version achieving scoring 57.1% on ARC
(Delta5.4%). We release our model, training recipes, and ablation studies to
facilitate further research in efficient model alignment, demonstrating that
careful adaptation of optimization dynamics can help bridge the capability gap
between small and large language models.
The instruction-tuned SmolTulu-DPO-1130 model, based on a SmolLM2-1.7B base, achieves state-of-the-art performance on sub-2B parameter models in tasks like instruction following and mathematical reasoning by optimizing learning rate to batch size ratios.
instruction-tuned SmolTulu-DPO-1130 SmolLM2-1.7B learning rate batch size ARC GSM8K HellaSwag IFEval performance efficient model alignment