title: Enabling Flexible Multi-LLM Integration for Scalable Knowledge
    Aggregation
summary: Large language models (LLMs) have shown remarkable promise but remain
  challenging to continually improve through traditional finetuning, particularly
  when integrating capabilities from other specialized LLMs. Popular methods like
  ensemble and weight merging require substantial memory and struggle to adapt to
  changing data environments. Recent efforts have transferred knowledge from
  multiple LLMs into a single target model; however, they suffer from
  interference and degraded performance among tasks, largely due to limited
  flexibility in candidate selection and training pipelines. To address these
  issues, we propose a framework that adaptively selects and aggregates knowledge
  from diverse LLMs to build a single, stronger model, avoiding the high memory
  overhead of ensemble and inflexible weight merging. Specifically, we design an
  adaptive selection network that identifies the most relevant source LLMs based
  on their scores, thereby reducing knowledge interference. We further propose a
  dynamic weighted fusion strategy that accounts for the inherent strengths of
  candidate LLMs, along with a feedback-driven loss function that prevents the
  selector from converging on a single subset of sources. Experimental results
  demonstrate that our method can enable a more stable and scalable knowledge
  aggregation process while reducing knowledge interference by up to 50% compared
  to existing approaches. Code is avaliable at
  https://github.com/ZLKong/LLM_Integration
paper_authors._id: 683d0ac47852d920b7dc3dc5
paper_authors.name: Zhenglun Kong
paper_authors._id: 683d0ac47852d920b7dc3dc6
paper_authors.name: Zheng Zhan
paper_authors._id: 683d0ac47852d920b7dc3dc7
paper_authors.name: Shiyue Hou
paper_authors._id: 683d0ac47852d920b7dc3dc8
paper_authors.name: Yifan Gong
paper_authors._id: 683d0ac47852d920b7dc3dc9
paper_authors.name: Xin Meng
paper_authors._id: 683d0ac47852d920b7dc3dca
paper_authors.name: Pengwei Sui
paper_authors._id: 683d0ac47852d920b7dc3dcb
paper_authors.name: Peiyan Dong
paper_authors._id: 683d0ac47852d920b7dc3dcc
paper_authors.name: Xuan Shen
paper_authors._id: 683d0ac47852d920b7dc3dcd
paper_authors.name: Zifeng Wang
paper_authors._id: 683d0ac47852d920b7dc3dce
paper_authors.name: Pu Zhao
paper_authors._id: 683d0ac47852d920b7dc3dcf
paper_authors.name: Hao Tang
paper_authors._id: 683d0ac47852d920b7dc3dd0
paper_authors.name: Stratis Ioannidis
paper_authors._id: 683d0ac47852d920b7dc3dd1
paper_authors.name: Yanzhi Wang
paper_title: Enabling Flexible Multi-LLM Integration for Scalable Knowledge
    Aggregation
paper_summary: Large language models (LLMs) have shown remarkable promise but remain
  challenging to continually improve through traditional finetuning, particularly
  when integrating capabilities from other specialized LLMs. Popular methods like
  ensemble and weight merging require substantial memory and struggle to adapt to
  changing data environments. Recent efforts have transferred knowledge from
  multiple LLMs into a single target model; however, they suffer from
  interference and degraded performance among tasks, largely due to limited
  flexibility in candidate selection and training pipelines. To address these
  issues, we propose a framework that adaptively selects and aggregates knowledge
  from diverse LLMs to build a single, stronger model, avoiding the high memory
  overhead of ensemble and inflexible weight merging. Specifically, we design an
  adaptive selection network that identifies the most relevant source LLMs based
  on their scores, thereby reducing knowledge interference. We further propose a
  dynamic weighted fusion strategy that accounts for the inherent strengths of
  candidate LLMs, along with a feedback-driven loss function that prevents the
  selector from converging on a single subset of sources. Experimental results
  demonstrate that our method can enable a more stable and scalable knowledge
  aggregation process while reducing knowledge interference by up to 50% compared
  to existing approaches. Code is avaliable at
  https://github.com/ZLKong/LLM_Integration
paper_ai_summary: A framework for adaptive selection and dynamic weighted fusion of knowledge from multiple LLMs reduces interference and improves scalability in knowledge aggregation.
paper_ai_keywords: large language models fine-tuning ensemble weight merging adaptive selection network dynamic weighted fusion feedback-driven loss function knowledge interference