SPAM: Spike-Aware Adam with Momentum Reset for Stable LLM Training
Large Language Models (LLMs) have demonstrated exceptional performance across
diverse tasks, yet their training remains highly resource-intensive and
susceptible to critical challenges such as training instability. A predominant
source of this instability stems from gradient and loss spikes, which disrupt
the learning process, often leading to costly interventions like checkpoint
recovery and experiment restarts, further amplifying inefficiencies. This paper
presents a comprehensive investigation into gradient spikes observed during LLM
training, revealing their prevalence across multiple architectures and
datasets. Our analysis shows that these spikes can be up to 1000times larger
than typical gradients, substantially deteriorating model performance. To
address this issue, we propose Spike-Aware Adam with Momentum Reset SPAM, a
novel optimizer designed to counteract gradient spikes through momentum reset
and spike-aware gradient clipping. Extensive experiments, including both
pre-training and fine-tuning, demonstrate that SPAM consistently surpasses Adam
and its variants across various tasks, including (1) LLM pre-training from 60M
to 1B, (2) 4-bit LLM pre-training,(3) reinforcement learning, and (4) Time
Series Forecasting. Additionally, SPAM facilitates memory-efficient training by
enabling sparse momentum, where only a subset of momentum terms are maintained
and updated. When operating under memory constraints, SPAM outperforms
state-of-the-art memory-efficient optimizers such as GaLore and Adam-Mini. Our
work underscores the importance of mitigating gradient spikes in LLM training
and introduces an effective optimization strategy that enhances both training
stability and resource efficiency at scale. Code is available at
https://github.com/TianjinYellow/SPAM-Optimizer.git
678646a9195de4bd4d6f75c2
Tianjin Huang
admin_assigned
2025-01-14T13:11:11.237Z
64cd4743a785f2043b32915e
/avatars/ba0b497a194dfea8449112d71fc67654.svg
Tianjin Huang
user
TianjinHuang
678646a9195de4bd4d6f75c3
Ziquan Zhu
678646a9195de4bd4d6f75c4
Gaojie Jin
678646a9195de4bd4d6f75c5
Lu Liu
678646a9195de4bd4d6f75c6
Zhangyang Wang
678646a9195de4bd4d6f75c7
Shiwei Liu
admin_assigned
2025-01-14T14:53:45.458Z
65b04d2291e63920a7898c9e
https://cdn-avatars.huggingface.co/v1/production/uploads/65b04d2291e63920a7898c9e/iUHs235G4bqK-KnH_94ti.jpeg
Liu
user
Shiweiliuiiiiiii
SPAM: Spike-Aware Adam with Momentum Reset for Stable LLM Training
Large Language Models (LLMs) have demonstrated exceptional performance across
diverse tasks, yet their training remains highly resource-intensive and
susceptible to critical challenges such as training instability. A predominant
source of this instability stems from gradient and loss spikes, which disrupt
the learning process, often leading to costly interventions like checkpoint
recovery and experiment restarts, further amplifying inefficiencies. This paper
presents a comprehensive investigation into gradient spikes observed during LLM
training, revealing their prevalence across multiple architectures and
datasets. Our analysis shows that these spikes can be up to 1000times larger
than typical gradients, substantially deteriorating model performance. To
address this issue, we propose Spike-Aware Adam with Momentum Reset SPAM, a
novel optimizer designed to counteract gradient spikes through momentum reset
and spike-aware gradient clipping. Extensive experiments, including both
pre-training and fine-tuning, demonstrate that SPAM consistently surpasses Adam
and its variants across various tasks, including (1) LLM pre-training from 60M
to 1B, (2) 4-bit LLM pre-training,(3) reinforcement learning, and (4) Time
Series Forecasting. Additionally, SPAM facilitates memory-efficient training by
enabling sparse momentum, where only a subset of momentum terms are maintained
and updated. When operating under memory constraints, SPAM outperforms
state-of-the-art memory-efficient optimizers such as GaLore and Adam-Mini. Our
work underscores the importance of mitigating gradient spikes in LLM training
and introduces an effective optimization strategy that enhances both training
stability and resource efficiency at scale. Code is available at
https://github.com/TianjinYellow/SPAM-Optimizer.git
SPAM optimizer addresses training instability in LLMs by mitigating gradient spikes, enhancing performance and efficiency across various tasks.
LLMs gradient spikes loss spikes Spike-Aware Adam with Momentum Reset SPAM momentum reset spike-aware gradient clipping memory-efficient training sparse momentum GaLore Adam-Mini