title: If You Can't Use Them, Recycle Them: Optimizing Merging at Scale
    Mitigates Performance Tradeoffs
summary: Model merging has shown great promise at combining expert models, but the
  benefit of merging is unclear when merging ``generalist'' models trained on
  many tasks. We explore merging in the context of large (sim100B) models, by
  recycling checkpoints that exhibit tradeoffs among different tasks.
  Such checkpoints are often created in the process of developing a frontier
  model, and many suboptimal ones are usually discarded. Given a pool of model
  checkpoints obtained from different training runs (e.g., different stages,
  objectives, hyperparameters, and data mixtures), which naturally show tradeoffs
  across different language capabilities (e.g., instruction following vs. code
  generation), we investigate whether merging can recycle such suboptimal models
  into a Pareto-optimal one. Our optimization algorithm tunes the weight of each
  checkpoint in a linear combination, resulting in a Pareto-optimal models that
  outperforms both individual models and merge-based baselines. Further analysis
  shows that good merges tend to include almost all checkpoints with with
  non-zero weights, indicating that even seemingly bad initial checkpoints can
  contribute to good final merges.
paper_authors._id: 6757f0c88cf3fb3bbeba6b2d
paper_authors.name: Muhammad Khalifa
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-10T10:25:02.075Z
paper_authors._id: 5f350fe67e5835433862161b
paper_authors.avatarUrl: /avatars/8aecfdea5202968b1099412800a152e0.svg
paper_authors.fullname: Muhammad Khalifa
paper_authors.type: user
paper_authors.user: mkhalifa
paper_authors._id: 6757f0c88cf3fb3bbeba6b2e
paper_authors.name: Yi-Chern Tan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-18T13:46:44.599Z
paper_authors._id: 675c5c1b2905333f65d810d5
paper_authors.avatarUrl: /avatars/f55d964457bb9f9e54f87d506167a821.svg
paper_authors.fullname: Yi-Chern Tan
paper_authors.type: user
paper_authors.user: yichern
paper_authors._id: 6757f0c88cf3fb3bbeba6b2f
paper_authors.name: Arash Ahmadian
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-18T13:46:37.630Z
paper_authors._id: 661b21f8ea926e8f86729e61
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/661b21f8ea926e8f86729e61/gM59piqaqjEu2IPbDvCbZ.jpeg
paper_authors.fullname: Arash Ahmadian
paper_authors.type: user
paper_authors.user: ArashAhmadian
paper_authors._id: 6757f0c88cf3fb3bbeba6b30
paper_authors.name: Tom Hosking
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-18T13:46:31.318Z
paper_authors._id: 6269415057cd241f40d6565c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6269415057cd241f40d6565c/ZJkTQDjU1dCejPu9BPfsB.jpeg
paper_authors.fullname: Tom Hosking
paper_authors.type: user
paper_authors.user: tomhosking
paper_authors._id: 6757f0c88cf3fb3bbeba6b31
paper_authors.name: Honglak Lee
paper_authors._id: 6757f0c88cf3fb3bbeba6b32
paper_authors.name: Lu Wang
paper_authors._id: 6757f0c88cf3fb3bbeba6b33
paper_authors.name: Ahmet Üstün
paper_authors._id: 6757f0c88cf3fb3bbeba6b34
paper_authors.name: Tom Sherborne
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-18T13:46:03.518Z
paper_authors._id: 5fae7fb3b8423e1d80b8a98f
paper_authors.avatarUrl: /avatars/1c0575701529ae0f15a6fed6834a473c.svg
paper_authors.fullname: Tom Sherborne
paper_authors.type: user
paper_authors.user: tomsherborne
paper_authors._id: 6757f0c88cf3fb3bbeba6b35
paper_authors.name: Matthias Gallé
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-18T13:45:54.518Z
paper_authors._id: 607fc2824ad99100d63ce35f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1634907979945-607fc2824ad99100d63ce35f.png
paper_authors.fullname: Matthias Gallé
paper_authors.type: user
paper_authors.user: mgalle
paper_title: If You Can't Use Them, Recycle Them: Optimizing Merging at Scale
    Mitigates Performance Tradeoffs
paper_summary: Model merging has shown great promise at combining expert models, but the
  benefit of merging is unclear when merging ``generalist'' models trained on
  many tasks. We explore merging in the context of large (sim100B) models, by
  recycling checkpoints that exhibit tradeoffs among different tasks.
  Such checkpoints are often created in the process of developing a frontier
  model, and many suboptimal ones are usually discarded. Given a pool of model
  checkpoints obtained from different training runs (e.g., different stages,
  objectives, hyperparameters, and data mixtures), which naturally show tradeoffs
  across different language capabilities (e.g., instruction following vs. code
  generation), we investigate whether merging can recycle such suboptimal models
  into a Pareto-optimal one. Our optimization algorithm tunes the weight of each
  checkpoint in a linear combination, resulting in a Pareto-optimal models that
  outperforms both individual models and merge-based baselines. Further analysis
  shows that good merges tend to include almost all checkpoints with with
  non-zero weights, indicating that even seemingly bad initial checkpoints can
  contribute to good final merges.
paper_ai_summary: Merging tradeoff-exhibiting checkpoints from large models can result in Pareto-optimal models that outperform individual models and baselines.
paper_ai_keywords: model merging checkpoints optimization algorithm Pareto-optimal language capabilities instruction following code generation linear combination