SemiEvol: Semi-supervised Fine-tuning for LLM Adaptation
Supervised fine-tuning (SFT) is crucial in adapting large language models
(LLMs) to a specific domain or task. However, only a limited amount of labeled
data is available in practical applications, which poses a severe challenge for
SFT in yielding satisfactory results. Therefore, a data-efficient framework
that can fully exploit labeled and unlabeled data for LLM fine-tuning is highly
anticipated. Towards this end, we introduce a semi-supervised fine-tuning
framework named SemiEvol for LLM adaptation from a propagate-and-select manner.
For knowledge propagation, SemiEvol adopts a bi-level approach, propagating
knowledge from labeled data to unlabeled data through both in-weight and
in-context methods. For knowledge selection, SemiEvol incorporates a
collaborative learning mechanism, selecting higher-quality pseudo-response
samples. We conducted experiments using GPT-4o-mini and Llama-3.1 on seven
general or domain-specific datasets, demonstrating significant improvements in
model performance on target data. Furthermore, we compared SemiEvol with SFT
and self-evolution methods, highlighting its practicality in hybrid data
scenarios.
671701fe9ec0ff4d1501fa6c
Junyu Luo
claimed_verified
2024-10-22T08:01:10.194Z
642da1cd99f3110ac27caca5
https://cdn-avatars.huggingface.co/v1/production/uploads/642da1cd99f3110ac27caca5/C1QJY3R_ZdaeANG1y8iW7.jpeg
junyu
user
luojunyu
671701fe9ec0ff4d1501fa6d
Xiao Luo
claimed_verified
2024-10-23T07:34:51.377Z
67103e13377b0d6540e54a78
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/pfpNTJG1A11ubZkukI-le.jpeg
PatriciaXiao
user
PatriciaXiao
671701fe9ec0ff4d1501fa6e
Xiusi Chen
admin_assigned
2024-10-22T09:36:49.362Z
6270ff726417aed8a7340c8b
/avatars/3f14913c55cc4fc78678ac43fb603e80.svg
Xiusi Chen
user
XtremSup
671701fe9ec0ff4d1501fa6f
Zhiping Xiao
671701fe9ec0ff4d1501fa70
Wei Ju
671701fe9ec0ff4d1501fa71
Ming Zhang
SemiEvol: Semi-supervised Fine-tuning for LLM Adaptation
Supervised fine-tuning (SFT) is crucial in adapting large language models
(LLMs) to a specific domain or task. However, only a limited amount of labeled
data is available in practical applications, which poses a severe challenge for
SFT in yielding satisfactory results. Therefore, a data-efficient framework
that can fully exploit labeled and unlabeled data for LLM fine-tuning is highly
anticipated. Towards this end, we introduce a semi-supervised fine-tuning
framework named SemiEvol for LLM adaptation from a propagate-and-select manner.
For knowledge propagation, SemiEvol adopts a bi-level approach, propagating
knowledge from labeled data to unlabeled data through both in-weight and
in-context methods. For knowledge selection, SemiEvol incorporates a
collaborative learning mechanism, selecting higher-quality pseudo-response
samples. We conducted experiments using GPT-4o-mini and Llama-3.1 on seven
general or domain-specific datasets, demonstrating significant improvements in
model performance on target data. Furthermore, we compared SemiEvol with SFT
and self-evolution methods, highlighting its practicality in hybrid data
scenarios.
A semi-supervised fine-tuning framework named SemiEvol enhances LLM adaptation using both labeled and unlabeled data, showing improved performance through bi-level knowledge propagation and collaborative learning.
supervised fine-tuning large language models semi-supervised fine-tuning data-efficient framework knowledge propagation bi-level approach in-weight in-context knowledge selection collaborative learning pseudo-response samples hybrid data scenarios