title: FuseChat: Knowledge Fusion of Chat Models
summary: While training large language models (LLMs) from scratch can indeed lead to
  models with distinct capabilities and strengths, it incurs substantial costs
  and may lead to redundancy in competencies. Knowledge fusion aims to integrate
  existing LLMs of diverse architectures and capabilities into a more potent LLM
  through lightweight continual training, thereby reducing the need for costly
  LLM development. In this work, we propose a new framework for the knowledge
  fusion of chat LLMs through two main stages, resulting in FuseChat. Firstly, we
  conduct pairwise knowledge fusion on source chat LLMs of varying structures and
  scales to create multiple target LLMs with identical structure and size via
  lightweight fine-tuning. During this process, a statistics-based token
  alignment approach is introduced as the cornerstone for fusing LLMs with
  different structures. Secondly, we merge these target LLMs within the parameter
  space, where we propose a novel method for determining the merging coefficients
  based on the magnitude of parameter updates before and after fine-tuning. We
  implement and validate FuseChat using six prominent chat LLMs with diverse
  architectures and scales, including OpenChat-3.5-7B, Starling-LM-7B-alpha,
  NH2-SOLAR-10.7B, InternLM2-Chat-20B, Mixtral-8x7B-Instruct, and
  Qwen-1.5-Chat-72B. Experimental results on two instruction-following
  benchmarks, AlpacaEval 2.0 and MT-Bench, demonstrate the superiority of
  FuseChat-7B over baselines of various sizes. Our model is even comparable to
  the larger Mixtral-8x7B-Instruct and approaches GPT-3.5-Turbo-1106 on MT-Bench.
  Our code, model weights, and data are public at
  https://github.com/fanqiwan/FuseAI.
paper_authors._id: 66bf2c76db33653b9008c1ea
paper_authors.name: Fanqi Wan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-16T14:20:55.954Z
paper_authors._id: 62ecbffd99112e99c5f7fded
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62ecbffd99112e99c5f7fded/U6iXAJbpm2vaC5qksEPiH.png
paper_authors.fullname: Fanqi Wan
paper_authors.type: user
paper_authors.user: Wanfq
paper_authors._id: 66bf2c76db33653b9008c1eb
paper_authors.name: Longguang Zhong
paper_authors._id: 66bf2c76db33653b9008c1ec
paper_authors.name: Ziyi Yang
paper_authors._id: 66bf2c76db33653b9008c1ed
paper_authors.name: Ruijun Chen
paper_authors._id: 66bf2c76db33653b9008c1ee
paper_authors.name: Xiaojun Quan
paper_title: FuseChat: Knowledge Fusion of Chat Models
paper_summary: While training large language models (LLMs) from scratch can indeed lead to
  models with distinct capabilities and strengths, it incurs substantial costs
  and may lead to redundancy in competencies. Knowledge fusion aims to integrate
  existing LLMs of diverse architectures and capabilities into a more potent LLM
  through lightweight continual training, thereby reducing the need for costly
  LLM development. In this work, we propose a new framework for the knowledge
  fusion of chat LLMs through two main stages, resulting in FuseChat. Firstly, we
  conduct pairwise knowledge fusion on source chat LLMs of varying structures and
  scales to create multiple target LLMs with identical structure and size via
  lightweight fine-tuning. During this process, a statistics-based token
  alignment approach is introduced as the cornerstone for fusing LLMs with
  different structures. Secondly, we merge these target LLMs within the parameter
  space, where we propose a novel method for determining the merging coefficients
  based on the magnitude of parameter updates before and after fine-tuning. We
  implement and validate FuseChat using six prominent chat LLMs with diverse
  architectures and scales, including OpenChat-3.5-7B, Starling-LM-7B-alpha,
  NH2-SOLAR-10.7B, InternLM2-Chat-20B, Mixtral-8x7B-Instruct, and
  Qwen-1.5-Chat-72B. Experimental results on two instruction-following
  benchmarks, AlpacaEval 2.0 and MT-Bench, demonstrate the superiority of
  FuseChat-7B over baselines of various sizes. Our model is even comparable to
  the larger Mixtral-8x7B-Instruct and approaches GPT-3.5-Turbo-1106 on MT-Bench.
  Our code, model weights, and data are public at
  https://github.com/fanqiwan/FuseAI.
paper_ai_summary: FuseChat integrates diverse chat LLMs using lightweight fine-tuning and parameter-space merging, resulting in a highly capable model competitive with larger LLMs.
paper_ai_keywords: knowledge fusion chat LLMs lightweight continual training fine-tuning token alignment parameter space parameter updates magnitude-based merging coefficients