Running
6
📚
🧬 RegMix: Data Mixture as Regression
Automatic data mixture method for large language model pre-training
-
-
RegMix: Data Mixture as Regression for Language Model Pre-training
Paper • 2407.01492 • Published • 34 -
sail/data-mixture-human-1b
Text Generation • Updated • 16 • 2 -
sail/data-mixture-pile-cc-1b
Text Generation • Updated • 17 • 2 -
sail/data-mixture-regmix-1b
Text Generation • Updated • 17 • 2 -
sail/data-mixture-doremi-1b
Text Generation • Updated • 14 • 2 -
sail/data-mixture-random-1b
Text Generation • Updated • 15 • 4 -
sail/regmix-data-sample
Viewer • Updated • 698k • 861 • 2 -
sail/regmix-data
Viewer • Updated • 13.7M • 10.4k • 3