FuseChat: Knowledge Fusion of Chat Models
While training large language models (LLMs) from scratch can indeed lead to
models with distinct capabilities and strengths, this approach incurs
substantial costs and may lead to potential redundancy in competencies. An
alternative strategy is to combine existing LLMs into a more robust LLM,
thereby diminishing the necessity for expensive pre-training. However, due to
the diverse architectures of LLMs, direct parameter blending proves to be
unfeasible. Recently, FuseLLM introduced the concept of knowledge
fusion to transfer the collective knowledge of multiple structurally varied
LLMs into a target LLM through lightweight continual training. In this report,
we extend the scalability and flexibility of the FuseLLM framework to
realize the fusion of chat LLMs, resulting in FuseChat.
FuseChat comprises two main stages. Firstly, we undertake knowledge
fusion for structurally and scale-varied source LLMs to derive multiple target
LLMs of identical structure and size via lightweight fine-tuning. Then, these
target LLMs are merged within the parameter space, wherein we propose a novel
method for determining the merging weights based on the variation ratio of
parameter matrices before and after fine-tuning. We validate our approach using
three prominent chat LLMs with diverse architectures and scales, namely
NH2-Mixtral-8x7B, NH2-Solar-10.7B, and
OpenChat-3.5-7B. Experimental results spanning various chat domains
demonstrate the superiority of \textsc{FuseChat-7B} across a broad
spectrum of chat LLMs at 7B and 34B scales, even surpassing GPT-3.5
(March) and approaching Mixtral-8x7B-Instruct. Our code, model
weights, and data are openly accessible at
https://github.com/fanqiwan/FuseLLM.
65dd6bc4e731928ad80885f7
Fanqi Wan
claimed_verified
2024-02-27T08:11:51.149Z
62ecbffd99112e99c5f7fded
https://cdn-avatars.huggingface.co/v1/production/uploads/62ecbffd99112e99c5f7fded/U6iXAJbpm2vaC5qksEPiH.png
Fanqi Wan
user
Wanfq
65dd6bc4e731928ad80885f8
Ziyi Yang
claimed_verified
2024-02-28T08:28:22.513Z
64c9b0f28d2d187c24d1e6c1
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/1CPnAaB3gsupdpiNWaoDc.png
ZiYi Yang
user
AALF
65dd6bc4e731928ad80885f9
Longguang Zhong
65dd6bc4e731928ad80885fa
Xiaojun Quan
admin_assigned
2024-02-27T09:28:38.967Z
63b57d75bda8d44adf2ff3ff
/avatars/8a387036758b2f7fc7d7529dea206669.svg
Xiaojun Quan
user
passerqxj
65dd6bc4e731928ad80885fb
Xinting Huang
claimed_verified
2024-05-14T13:51:52.984Z
63c227a6c58fcfeac18dec07
/avatars/47df92c937cfaf8d726c7e28cf352eb5.svg
tim huang
user
timmyhhh
65dd6bc4e731928ad80885fc
Wei Bi
FuseChat: Knowledge Fusion of Chat Models
While training large language models (LLMs) from scratch can indeed lead to
models with distinct capabilities and strengths, this approach incurs
substantial costs and may lead to potential redundancy in competencies. An
alternative strategy is to combine existing LLMs into a more robust LLM,
thereby diminishing the necessity for expensive pre-training. However, due to
the diverse architectures of LLMs, direct parameter blending proves to be
unfeasible. Recently, FuseLLM introduced the concept of knowledge
fusion to transfer the collective knowledge of multiple structurally varied
LLMs into a target LLM through lightweight continual training. In this report,
we extend the scalability and flexibility of the FuseLLM framework to
realize the fusion of chat LLMs, resulting in FuseChat.
FuseChat comprises two main stages. Firstly, we undertake knowledge
fusion for structurally and scale-varied source LLMs to derive multiple target
LLMs of identical structure and size via lightweight fine-tuning. Then, these
target LLMs are merged within the parameter space, wherein we propose a novel
method for determining the merging weights based on the variation ratio of
parameter matrices before and after fine-tuning. We validate our approach using
three prominent chat LLMs with diverse architectures and scales, namely
NH2-Mixtral-8x7B, NH2-Solar-10.7B, and
OpenChat-3.5-7B. Experimental results spanning various chat domains
demonstrate the superiority of \textsc{FuseChat-7B} across a broad
spectrum of chat LLMs at 7B and 34B scales, even surpassing GPT-3.5
(March) and approaching Mixtral-8x7B-Instruct. Our code, model
weights, and data are openly accessible at
https://github.com/fanqiwan/FuseLLM.
FuseChat extends the FuseLLM framework for knowledge fusion of chat LLMs through lightweight fine-tuning and parameter merging, achieving superior performance across various domains.
knowledge fusion fine-tuning parameter merging variation ratio chat LLMs Mixtral-8x7B-Instruct GPT-3.5