If You Can't Use Them, Recycle Them: Optimizing Merging at Scale
  Mitigates Performance Tradeoffs
Model merging has shown great promise at combining expert models, but the
benefit of merging is unclear when merging ``generalist'' models trained on
many tasks. We explore merging in the context of large (sim100B) models, by
recycling checkpoints that exhibit tradeoffs among different tasks.
Such checkpoints are often created in the process of developing a frontier
model, and many suboptimal ones are usually discarded. Given a pool of model
checkpoints obtained from different training runs (e.g., different stages,
objectives, hyperparameters, and data mixtures), which naturally show tradeoffs
across different language capabilities (e.g., instruction following vs. code
generation), we investigate whether merging can recycle such suboptimal models
into a Pareto-optimal one. Our optimization algorithm tunes the weight of each
checkpoint in a linear combination, resulting in a Pareto-optimal models that
outperforms both individual models and merge-based baselines. Further analysis
shows that good merges tend to include almost all checkpoints with with
non-zero weights, indicating that even seemingly bad initial checkpoints can
contribute to good final merges.
6757f0c88cf3fb3bbeba6b2d
Muhammad Khalifa
claimed_verified
2024-12-10T10:25:02.075Z
5f350fe67e5835433862161b
/avatars/8aecfdea5202968b1099412800a152e0.svg
Muhammad Khalifa
user
mkhalifa
6757f0c88cf3fb3bbeba6b2e
Yi-Chern Tan
admin_assigned
2024-12-18T13:46:44.599Z
675c5c1b2905333f65d810d5
/avatars/f55d964457bb9f9e54f87d506167a821.svg
Yi-Chern Tan
user
yichern
6757f0c88cf3fb3bbeba6b2f
Arash Ahmadian
admin_assigned
2024-12-18T13:46:37.630Z
661b21f8ea926e8f86729e61
https://cdn-avatars.huggingface.co/v1/production/uploads/661b21f8ea926e8f86729e61/gM59piqaqjEu2IPbDvCbZ.jpeg
Arash Ahmadian
user
ArashAhmadian
6757f0c88cf3fb3bbeba6b30
Tom Hosking
admin_assigned
2024-12-18T13:46:31.318Z
6269415057cd241f40d6565c
https://cdn-avatars.huggingface.co/v1/production/uploads/6269415057cd241f40d6565c/ZJkTQDjU1dCejPu9BPfsB.jpeg
Tom Hosking
user
tomhosking
6757f0c88cf3fb3bbeba6b31
Honglak Lee
6757f0c88cf3fb3bbeba6b32
Lu Wang
6757f0c88cf3fb3bbeba6b33
Ahmet Üstün
6757f0c88cf3fb3bbeba6b34
Tom Sherborne
admin_assigned
2024-12-18T13:46:03.518Z
5fae7fb3b8423e1d80b8a98f
/avatars/1c0575701529ae0f15a6fed6834a473c.svg
Tom Sherborne
user
tomsherborne
6757f0c88cf3fb3bbeba6b35
Matthias Gallé
admin_assigned
2024-12-18T13:45:54.518Z
607fc2824ad99100d63ce35f
https://cdn-avatars.huggingface.co/v1/production/uploads/1634907979945-607fc2824ad99100d63ce35f.png
Matthias Gallé
user
mgalle
If You Can't Use Them, Recycle Them: Optimizing Merging at Scale
  Mitigates Performance Tradeoffs
Model merging has shown great promise at combining expert models, but the
benefit of merging is unclear when merging ``generalist'' models trained on
many tasks. We explore merging in the context of large (sim100B) models, by
recycling checkpoints that exhibit tradeoffs among different tasks.
Such checkpoints are often created in the process of developing a frontier
model, and many suboptimal ones are usually discarded. Given a pool of model
checkpoints obtained from different training runs (e.g., different stages,
objectives, hyperparameters, and data mixtures), which naturally show tradeoffs
across different language capabilities (e.g., instruction following vs. code
generation), we investigate whether merging can recycle such suboptimal models
into a Pareto-optimal one. Our optimization algorithm tunes the weight of each
checkpoint in a linear combination, resulting in a Pareto-optimal models that
outperforms both individual models and merge-based baselines. Further analysis
shows that good merges tend to include almost all checkpoints with with
non-zero weights, indicating that even seemingly bad initial checkpoints can
contribute to good final merges.
Merging tradeoff-exhibiting checkpoints from large models can result in Pareto-optimal models that outperform individual models and baselines.
model merging checkpoints optimization algorithm Pareto-optimal language capabilities instruction following code generation linear combination