view article Article SmolLM3: smol, multilingual, long-context reasoner By loubnabnl and 22 others • 20 days ago • 585
Running 1.01k 1.01k FineWeb: decanting the web for the finest text data at scale 🍷 Generate high-quality web text data for LLM training
Running 2.84k 2.84k The Ultra-Scale Playbook 🌌 The ultimate guide to training LLM on large GPU Clusters
view article Article DualPipe Explained: A Comprehensive Guide to DualPipe That Anyone Can Understand—Even Without a Distributed Training Background By NormalUhr • Feb 28 • 9
Zephyr ORPO Collection Models and datasets to align LLMs with Odds Ratio Preference Optimisation (ORPO). Recipes here: https://github.com/huggingface/alignment-handbook • 3 items • Updated Apr 12, 2024 • 17