title: MUSCLE: A Model Update Strategy for Compatible LLM Evolution
summary: Large Language Models (LLMs) are frequently updated due to data or
  architecture changes to improve their performance. When updating models,
  developers often focus on increasing overall performance metrics with less
  emphasis on being compatible with previous model versions. However, users often
  build a mental model of the functionality and capabilities of a particular
  machine learning model they are interacting with. They have to adapt their
  mental model with every update -- a draining task that can lead to user
  dissatisfaction. In practice, fine-tuned downstream task adapters rely on
  pretrained LLM base models. When these base models are updated, these
  user-facing downstream task models experience instance regression or negative
  flips -- previously correct instances are now predicted incorrectly. This
  happens even when the downstream task training procedures remain identical. Our
  work aims to provide seamless model updates to a user in two ways. First, we
  provide evaluation metrics for a notion of compatibility to prior model
  versions, specifically for generative tasks but also applicable for
  discriminative tasks. We observe regression and inconsistencies between
  different model versions on a diverse set of tasks and model updates. Second,
  we propose a training strategy to minimize the number of inconsistencies in
  model updates, involving training of a compatibility model that can enhance
  task fine-tuned language models. We reduce negative flips -- instances where a
  prior model version was correct, but a new model incorrect -- by up to 40% from
  Llama 1 to Llama 2.
paper_authors._id: 66948ba1df5d51613344d461
paper_authors.name: Jessica Echterhoff
paper_authors._id: 66948ba1df5d51613344d462
paper_authors.name: Fartash Faghri
paper_authors._id: 66948ba1df5d51613344d463
paper_authors.name: Raviteja Vemulapalli
paper_authors._id: 66948ba1df5d51613344d464
paper_authors.name: Ting-Yao Hu
paper_authors._id: 66948ba1df5d51613344d465
paper_authors.name: Chun-Liang Li
paper_authors._id: 66948ba1df5d51613344d466
paper_authors.name: Oncel Tuzel
paper_authors._id: 66948ba1df5d51613344d467
paper_authors.name: Hadi Pouransari
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-07-15T02:38:26.354Z
paper_authors._id: 653a8e65c75dc136bfb5b0f8
paper_authors.avatarUrl: /avatars/ef252794236ce0fe2debf12773c95bb2.svg
paper_authors.fullname: Hadi Pouransari
paper_authors.type: user
paper_authors.user: hpouransari
paper_title: MUSCLE: A Model Update Strategy for Compatible LLM Evolution
paper_summary: Large Language Models (LLMs) are frequently updated due to data or
  architecture changes to improve their performance. When updating models,
  developers often focus on increasing overall performance metrics with less
  emphasis on being compatible with previous model versions. However, users often
  build a mental model of the functionality and capabilities of a particular
  machine learning model they are interacting with. They have to adapt their
  mental model with every update -- a draining task that can lead to user
  dissatisfaction. In practice, fine-tuned downstream task adapters rely on
  pretrained LLM base models. When these base models are updated, these
  user-facing downstream task models experience instance regression or negative
  flips -- previously correct instances are now predicted incorrectly. This
  happens even when the downstream task training procedures remain identical. Our
  work aims to provide seamless model updates to a user in two ways. First, we
  provide evaluation metrics for a notion of compatibility to prior model
  versions, specifically for generative tasks but also applicable for
  discriminative tasks. We observe regression and inconsistencies between
  different model versions on a diverse set of tasks and model updates. Second,
  we propose a training strategy to minimize the number of inconsistencies in
  model updates, involving training of a compatibility model that can enhance
  task fine-tuned language models. We reduce negative flips -- instances where a
  prior model version was correct, but a new model incorrect -- by up to 40% from
  Llama 1 to Llama 2.
paper_ai_keywords: Large Language Models (LLMs) fine-tuned downstream task adapters pretrained LLM base models instance regression negative flips compatibility metrics generative tasks discriminative tasks compatibility model negative flips