title: FuseChat: Knowledge Fusion of Chat Models
summary: While training large language models (LLMs) from scratch can indeed lead to
  models with distinct capabilities and strengths, this approach incurs
  substantial costs and may lead to potential redundancy in competencies. An
  alternative strategy is to combine existing LLMs into a more robust LLM,
  thereby diminishing the necessity for expensive pre-training. However, due to
  the diverse architectures of LLMs, direct parameter blending proves to be
  unfeasible. Recently, FuseLLM introduced the concept of knowledge
  fusion to transfer the collective knowledge of multiple structurally varied
  LLMs into a target LLM through lightweight continual training. In this report,
  we extend the scalability and flexibility of the FuseLLM framework to
  realize the fusion of chat LLMs, resulting in FuseChat.
  FuseChat comprises two main stages. Firstly, we undertake knowledge
  fusion for structurally and scale-varied source LLMs to derive multiple target
  LLMs of identical structure and size via lightweight fine-tuning. Then, these
  target LLMs are merged within the parameter space, wherein we propose a novel
  method for determining the merging weights based on the variation ratio of
  parameter matrices before and after fine-tuning. We validate our approach using
  three prominent chat LLMs with diverse architectures and scales, namely
  NH2-Mixtral-8x7B, NH2-Solar-10.7B, and
  OpenChat-3.5-7B. Experimental results spanning various chat domains
  demonstrate the superiority of \textsc{FuseChat-7B} across a broad
  spectrum of chat LLMs at 7B and 34B scales, even surpassing GPT-3.5
  (March) and approaching Mixtral-8x7B-Instruct. Our code, model
  weights, and data are openly accessible at
  https://github.com/fanqiwan/FuseLLM.
paper_authors._id: 65dd6bc4e731928ad80885f7
paper_authors.name: Fanqi Wan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-27T08:11:51.149Z
paper_authors._id: 62ecbffd99112e99c5f7fded
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62ecbffd99112e99c5f7fded/U6iXAJbpm2vaC5qksEPiH.png
paper_authors.fullname: Fanqi Wan
paper_authors.type: user
paper_authors.user: Wanfq
paper_authors._id: 65dd6bc4e731928ad80885f8
paper_authors.name: Ziyi Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-28T08:28:22.513Z
paper_authors._id: 64c9b0f28d2d187c24d1e6c1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/1CPnAaB3gsupdpiNWaoDc.png
paper_authors.fullname: ZiYi Yang
paper_authors.type: user
paper_authors.user: AALF
paper_authors._id: 65dd6bc4e731928ad80885f9
paper_authors.name: Longguang Zhong
paper_authors._id: 65dd6bc4e731928ad80885fa
paper_authors.name: Xiaojun Quan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-27T09:28:38.967Z
paper_authors._id: 63b57d75bda8d44adf2ff3ff
paper_authors.avatarUrl: /avatars/8a387036758b2f7fc7d7529dea206669.svg
paper_authors.fullname: Xiaojun Quan
paper_authors.type: user
paper_authors.user: passerqxj
paper_authors._id: 65dd6bc4e731928ad80885fb
paper_authors.name: Xinting Huang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-05-14T13:51:52.984Z
paper_authors._id: 63c227a6c58fcfeac18dec07
paper_authors.avatarUrl: /avatars/47df92c937cfaf8d726c7e28cf352eb5.svg
paper_authors.fullname: tim huang
paper_authors.type: user
paper_authors.user: timmyhhh
paper_authors._id: 65dd6bc4e731928ad80885fc
paper_authors.name: Wei Bi
paper_title: FuseChat: Knowledge Fusion of Chat Models
paper_summary: While training large language models (LLMs) from scratch can indeed lead to
  models with distinct capabilities and strengths, this approach incurs
  substantial costs and may lead to potential redundancy in competencies. An
  alternative strategy is to combine existing LLMs into a more robust LLM,
  thereby diminishing the necessity for expensive pre-training. However, due to
  the diverse architectures of LLMs, direct parameter blending proves to be
  unfeasible. Recently, FuseLLM introduced the concept of knowledge
  fusion to transfer the collective knowledge of multiple structurally varied
  LLMs into a target LLM through lightweight continual training. In this report,
  we extend the scalability and flexibility of the FuseLLM framework to
  realize the fusion of chat LLMs, resulting in FuseChat.
  FuseChat comprises two main stages. Firstly, we undertake knowledge
  fusion for structurally and scale-varied source LLMs to derive multiple target
  LLMs of identical structure and size via lightweight fine-tuning. Then, these
  target LLMs are merged within the parameter space, wherein we propose a novel
  method for determining the merging weights based on the variation ratio of
  parameter matrices before and after fine-tuning. We validate our approach using
  three prominent chat LLMs with diverse architectures and scales, namely
  NH2-Mixtral-8x7B, NH2-Solar-10.7B, and
  OpenChat-3.5-7B. Experimental results spanning various chat domains
  demonstrate the superiority of \textsc{FuseChat-7B} across a broad
  spectrum of chat LLMs at 7B and 34B scales, even surpassing GPT-3.5
  (March) and approaching Mixtral-8x7B-Instruct. Our code, model
  weights, and data are openly accessible at
  https://github.com/fanqiwan/FuseLLM.
paper_ai_keywords: large language models (LLMs) parameter blending robust LLM lightweight continual training FuseLLM knowledge fusion structurally varied LLMs target LLMs fine-tuning parameter space merging weights parameter matrices variation ratio chat LLMs near-identical structure \textsc{GenFuseChat} knowledge fusion \textsc{FuseChat} NH2-Mixtral-8x7B NH2-Solar-10.7B OpenChat-3.5-7B GPT-3.5 (March) Mixtral-8x7B-Instruct chat domains \textsc{FuseChat}-7B \textsc{FuseChat}-34B