title: RobustFT: Robust Supervised Fine-tuning for Large Language Models under
    Noisy Response
summary: Supervised fine-tuning (SFT) plays a crucial role in adapting large language
  models (LLMs) to specific domains or tasks. However, as demonstrated by
  empirical experiments, the collected data inevitably contains noise in
  practical applications, which poses significant challenges to model performance
  on downstream tasks. Therefore, there is an urgent need for a noise-robust SFT
  framework to enhance model capabilities in downstream tasks. To address this
  challenge, we introduce a robust SFT framework (RobustFT) that performs noise
  detection and relabeling on downstream task data. For noise identification, our
  approach employs a multi-expert collaborative system with inference-enhanced
  models to achieve superior noise detection. In the denoising phase, we utilize
  a context-enhanced strategy, which incorporates the most relevant and confident
  knowledge followed by careful assessment to generate reliable annotations.
  Additionally, we introduce an effective data selection mechanism based on
  response entropy, ensuring only high-quality samples are retained for
  fine-tuning. Extensive experiments conducted on multiple LLMs across five
  datasets demonstrate RobustFT's exceptional performance in noisy scenarios.
paper_authors._id: 676a2354463437b5e1217e15
paper_authors.name: Junyu Luo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-30T19:35:39.138Z
paper_authors._id: 642da1cd99f3110ac27caca5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/642da1cd99f3110ac27caca5/C1QJY3R_ZdaeANG1y8iW7.jpeg
paper_authors.fullname: junyu
paper_authors.type: user
paper_authors.user: luojunyu
paper_authors._id: 676a2354463437b5e1217e16
paper_authors.name: Xiao Luo
paper_authors._id: 676a2354463437b5e1217e17
paper_authors.name: Kaize Ding
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-12-24T02:58:28.896Z
paper_authors._id: 665e2f9301ca1c80a0a311d2
paper_authors.avatarUrl: /avatars/67c88b55b580e6db74df4d0091197cea.svg
paper_authors.fullname: Kaize Ding
paper_authors.type: user
paper_authors.user: kaize0409
paper_authors._id: 676a2354463437b5e1217e18
paper_authors.name: Jingyang Yuan
paper_authors._id: 676a2354463437b5e1217e19
paper_authors.name: Zhiping Xiao
paper_authors._id: 676a2354463437b5e1217e1a
paper_authors.name: Ming Zhang
paper_title: RobustFT: Robust Supervised Fine-tuning for Large Language Models under
    Noisy Response
paper_summary: Supervised fine-tuning (SFT) plays a crucial role in adapting large language
  models (LLMs) to specific domains or tasks. However, as demonstrated by
  empirical experiments, the collected data inevitably contains noise in
  practical applications, which poses significant challenges to model performance
  on downstream tasks. Therefore, there is an urgent need for a noise-robust SFT
  framework to enhance model capabilities in downstream tasks. To address this
  challenge, we introduce a robust SFT framework (RobustFT) that performs noise
  detection and relabeling on downstream task data. For noise identification, our
  approach employs a multi-expert collaborative system with inference-enhanced
  models to achieve superior noise detection. In the denoising phase, we utilize
  a context-enhanced strategy, which incorporates the most relevant and confident
  knowledge followed by careful assessment to generate reliable annotations.
  Additionally, we introduce an effective data selection mechanism based on
  response entropy, ensuring only high-quality samples are retained for
  fine-tuning. Extensive experiments conducted on multiple LLMs across five
  datasets demonstrate RobustFT's exceptional performance in noisy scenarios.
paper_ai_summary: A noise-robust supervised fine-tuning framework enhances large language models' performance in downstream tasks by detecting and relabeling noisy data.
paper_ai_keywords: supervised fine-tuning large language models noise-robust framework multi-expert collaborative system inference-enhanced models context-enhanced strategy response entropy data selection mechanism