title: SoRFT: Issue Resolving with Subtask-oriented Reinforced Fine-Tuning
summary: Mainstream issue-resolving frameworks predominantly rely on commercial
  models, leading to high costs and privacy concerns. Existing training
  approaches for issue resolving struggle with poor generalization and fail to
  fully leverage open-source development resources. We propose Subtask-oriented
  Reinforced Fine-Tuning (SoRFT), a novel training approach to enhance the issue
  resolving capability of LLMs. We decomposes issue resolving into structured
  subtasks: file localization, function localization, line localization, and code
  edit generation. SoRFT consists of two training stages: (1) rejection-sampled
  supervised fine-tuning, Chain of Thought (CoT) data is filtered using
  ground-truth before fine-tuning the LLM, and (2) rule-based reinforcement
  learning, which leverages PPO with ground-truth based rewards. We evaluate the
  SoRFT-trained model on SWE-Bench Verified and SWE-Bench Lite, achieving
  state-of-the-art (SOTA) performance among open-source models (e.g., resolve
  21.4% issues on SWE-Bench Verified with SoRFT-Qwen-7B). The experimental
  results demonstrate that SoRFT significantly enhances issue-resolving
  performance, improves model generalization, and provides a cost-efficient
  alternative to commercial models.
paper_authors._id: 67c12de08cd49ca63e230b99
paper_authors.name: Zexiong Ma
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-28T09:28:35.503Z
paper_authors._id: 654da66fb36f85a025bc24b6
paper_authors.avatarUrl: /avatars/e5542856ab4bf1845e8f546b5f17cd99.svg
paper_authors.fullname: Zexiong Ma
paper_authors.type: user
paper_authors.user: mizersy
paper_authors._id: 67c12de08cd49ca63e230b9a
paper_authors.name: Chao Peng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-28T15:13:55.041Z
paper_authors._id: 64425a502f4abae43fc0446c
paper_authors.avatarUrl: /avatars/7448a8d024813d8a20e09c162a189304.svg
paper_authors.fullname: Chao Peng
paper_authors.type: user
paper_authors.user: pengchao
paper_authors._id: 67c12de08cd49ca63e230b9b
paper_authors.name: Pengfei Gao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-28T12:40:45.303Z
paper_authors._id: 663f2dde4aeb9c177297fbd8
paper_authors.avatarUrl: /avatars/ddc143481d2af893c9cdff1a33ccda28.svg
paper_authors.fullname: PengFei
paper_authors.type: user
paper_authors.user: PengFeiGao
paper_authors._id: 67c12de08cd49ca63e230b9c
paper_authors.name: Xiangxin Meng
paper_authors._id: 67c12de08cd49ca63e230b9d
paper_authors.name: Yanzhen Zou
paper_authors._id: 67c12de08cd49ca63e230b9e
paper_authors.name: Bing Xie
paper_title: SoRFT: Issue Resolving with Subtask-oriented Reinforced Fine-Tuning
paper_summary: Mainstream issue-resolving frameworks predominantly rely on commercial
  models, leading to high costs and privacy concerns. Existing training
  approaches for issue resolving struggle with poor generalization and fail to
  fully leverage open-source development resources. We propose Subtask-oriented
  Reinforced Fine-Tuning (SoRFT), a novel training approach to enhance the issue
  resolving capability of LLMs. We decomposes issue resolving into structured
  subtasks: file localization, function localization, line localization, and code
  edit generation. SoRFT consists of two training stages: (1) rejection-sampled
  supervised fine-tuning, Chain of Thought (CoT) data is filtered using
  ground-truth before fine-tuning the LLM, and (2) rule-based reinforcement
  learning, which leverages PPO with ground-truth based rewards. We evaluate the
  SoRFT-trained model on SWE-Bench Verified and SWE-Bench Lite, achieving
  state-of-the-art (SOTA) performance among open-source models (e.g., resolve
  21.4% issues on SWE-Bench Verified with SoRFT-Qwen-7B). The experimental
  results demonstrate that SoRFT significantly enhances issue-resolving
  performance, improves model generalization, and provides a cost-efficient
  alternative to commercial models.
paper_ai_summary: Subtask-oriented Reinforced Fine-Tuning (SoRFT) enhances issue-resolving performance of LLMs through structured subtasks and a combination of supervised fine-tuning and rule-based reinforcement learning with PPO.
paper_ai_keywords: Subtask-oriented Reinforced Fine-Tuning SoRFT rejection-sampled supervised fine-tuning Chain of Thought data rule-based reinforcement learning Proximal Policy Optimization PPO SWE-Bench Verified SWE-Bench Lite