SlimQwen: Exploring the Pruning and Distillation in Large MoE Model Pre-training
Paper • 2605.08738 • Published • 13
Liangyu Wang
ly4096
AI & ML interests
Efficient reinforcement learning (RL) for LLMs reasoning
Distributed training and inference of LLMs
Efficient algorithm and infrastructure design for LLMs
Recent Activity
upvoted a paper 8 days ago
SlimQwen: Exploring the Pruning and Distillation in Large MoE Model Pre-training authored a paper 3 months ago
Canzona: A Unified, Asynchronous, and Load-Balanced Framework for Distributed Matrix-based Optimizers submitted a paper 3 months ago
Canzona: A Unified, Asynchronous, and Load-Balanced Framework for Distributed Matrix-based OptimizersOrganizations
upvoted a paper 8 days ago
upvoted 3
papers 3 months ago
Canzona: A Unified, Asynchronous, and Load-Balanced Framework for Distributed Matrix-based Optimizers
Paper • 2602.06079 • Published • 21
FlashDP: Private Training Large Language Models with Efficient DP-SGD
Paper • 2507.01154 • Published • 1
Infinite Sampling: Efficient and Stable Grouped RL Training for Large Language Models
Paper • 2506.22950 • Published • 1