Pipeline Parallelism - a QPHutu Collection

QPHutu 's Collections

LLM Pretraining

Pipeline Parallelism

Pipeline Parallelism

updated Mar 5

Zero Bubble Pipeline Parallelism

Paper • 2401.10241 • Published Nov 30, 2023 • 25
Pipeline Parallelism with Controllable Memory

Paper • 2405.15362 • Published May 24, 2024 • 3
Balancing Pipeline Parallelism with Vocabulary Parallelism

Paper • 2411.05288 • Published Nov 8, 2024 • 20
PipeOffload: Improving Scalability of Pipeline Parallelism with Memory Optimization

Paper • 2503.01328 • Published Mar 3 • 16