RobustFT: Robust Supervised Fine-tuning for Large Language Models under
  Noisy Response
Supervised fine-tuning (SFT) plays a crucial role in adapting large language
models (LLMs) to specific domains or tasks. However, as demonstrated by
empirical experiments, the collected data inevitably contains noise in
practical applications, which poses significant challenges to model performance
on downstream tasks. Therefore, there is an urgent need for a noise-robust SFT
framework to enhance model capabilities in downstream tasks. To address this
challenge, we introduce a robust SFT framework (RobustFT) that performs noise
detection and relabeling on downstream task data. For noise identification, our
approach employs a multi-expert collaborative system with inference-enhanced
models to achieve superior noise detection. In the denoising phase, we utilize
a context-enhanced strategy, which incorporates the most relevant and confident
knowledge followed by careful assessment to generate reliable annotations.
Additionally, we introduce an effective data selection mechanism based on
response entropy, ensuring only high-quality samples are retained for
fine-tuning. Extensive experiments conducted on multiple LLMs across five
datasets demonstrate RobustFT's exceptional performance in noisy scenarios.
676a2354463437b5e1217e15
Junyu Luo
claimed_verified
2024-12-30T19:35:39.138Z
642da1cd99f3110ac27caca5
https://cdn-avatars.huggingface.co/v1/production/uploads/642da1cd99f3110ac27caca5/C1QJY3R_ZdaeANG1y8iW7.jpeg
junyu
user
luojunyu
676a2354463437b5e1217e16
Xiao Luo
676a2354463437b5e1217e17
Kaize Ding
extracted_pending
2024-12-24T02:58:28.896Z
665e2f9301ca1c80a0a311d2
/avatars/67c88b55b580e6db74df4d0091197cea.svg
Kaize Ding
user
kaize0409
676a2354463437b5e1217e18
Jingyang Yuan
676a2354463437b5e1217e19
Zhiping Xiao
676a2354463437b5e1217e1a
Ming Zhang
RobustFT: Robust Supervised Fine-tuning for Large Language Models under
  Noisy Response
Supervised fine-tuning (SFT) plays a crucial role in adapting large language
models (LLMs) to specific domains or tasks. However, as demonstrated by
empirical experiments, the collected data inevitably contains noise in
practical applications, which poses significant challenges to model performance
on downstream tasks. Therefore, there is an urgent need for a noise-robust SFT
framework to enhance model capabilities in downstream tasks. To address this
challenge, we introduce a robust SFT framework (RobustFT) that performs noise
detection and relabeling on downstream task data. For noise identification, our
approach employs a multi-expert collaborative system with inference-enhanced
models to achieve superior noise detection. In the denoising phase, we utilize
a context-enhanced strategy, which incorporates the most relevant and confident
knowledge followed by careful assessment to generate reliable annotations.
Additionally, we introduce an effective data selection mechanism based on
response entropy, ensuring only high-quality samples are retained for
fine-tuning. Extensive experiments conducted on multiple LLMs across five
datasets demonstrate RobustFT's exceptional performance in noisy scenarios.
A noise-robust supervised fine-tuning framework enhances large language models' performance in downstream tasks by detecting and relabeling noisy data.
supervised fine-tuning large language models noise-robust framework multi-expert collaborative system inference-enhanced models context-enhanced strategy response entropy data selection mechanism