Selecting and Merging: Towards Adaptable and Scalable Named Entity
  Recognition with Large Language Models
Supervised fine-tuning (SFT) is widely used to align large language models
(LLMs) with information extraction (IE) tasks, such as named entity recognition
(NER). However, annotating such fine-grained labels and training
domain-specific models is costly. Existing works typically train a unified
model across multiple domains, but such approaches lack adaptation and
scalability since not all training data benefits target domains and scaling
trained models remains challenging. We propose the SaM framework, which
dynamically Selects and Merges expert models at inference time. Specifically,
for a target domain, we select domain-specific experts pre-trained on existing
domains based on (i) domain similarity to the target domain and (ii)
performance on sampled instances, respectively. The experts are then merged to
create task-specific models optimized for the target domain. By dynamically
merging experts beneficial to target domains, we improve generalization across
various domains without extra training. Additionally, experts can be added or
removed conveniently, leading to great scalability. Extensive experiments on
multiple benchmarks demonstrate our framework's effectiveness, which
outperforms the unified model by an average of 10%. We further provide insights
into potential improvements, practical experience, and extensions of our
framework.
6867836c11736b002cf34d41
Zhuojun Ding
6867836c11736b002cf34d42
Wei Wei
6867836c11736b002cf34d43
Chenghao Fan
claimed_verified
2025-07-04T07:52:43.972Z
641aa5e391e3376a057bbd4c
/avatars/5818797f27444fde078b503774ee081c.svg
Chenghao Fan
user
Facico
Selecting and Merging: Towards Adaptable and Scalable Named Entity
  Recognition with Large Language Models
Supervised fine-tuning (SFT) is widely used to align large language models
(LLMs) with information extraction (IE) tasks, such as named entity recognition
(NER). However, annotating such fine-grained labels and training
domain-specific models is costly. Existing works typically train a unified
model across multiple domains, but such approaches lack adaptation and
scalability since not all training data benefits target domains and scaling
trained models remains challenging. We propose the SaM framework, which
dynamically Selects and Merges expert models at inference time. Specifically,
for a target domain, we select domain-specific experts pre-trained on existing
domains based on (i) domain similarity to the target domain and (ii)
performance on sampled instances, respectively. The experts are then merged to
create task-specific models optimized for the target domain. By dynamically
merging experts beneficial to target domains, we improve generalization across
various domains without extra training. Additionally, experts can be added or
removed conveniently, leading to great scalability. Extensive experiments on
multiple benchmarks demonstrate our framework's effectiveness, which
outperforms the unified model by an average of 10%. We further provide insights
into potential improvements, practical experience, and extensions of our
framework.
The SaM framework dynamically selects and merges domain-specific expert models to improve generalization and scalability in information extraction tasks without additional training.
supervised fine-tuning large language models information extraction named entity recognition domain-specific models unified model domain similarity task-specific models generalization scalability