SoRFT: Issue Resolving with Subtask-oriented Reinforced Fine-Tuning
Mainstream issue-resolving frameworks predominantly rely on commercial
models, leading to high costs and privacy concerns. Existing training
approaches for issue resolving struggle with poor generalization and fail to
fully leverage open-source development resources. We propose Subtask-oriented
Reinforced Fine-Tuning (SoRFT), a novel training approach to enhance the issue
resolving capability of LLMs. We decomposes issue resolving into structured
subtasks: file localization, function localization, line localization, and code
edit generation. SoRFT consists of two training stages: (1) rejection-sampled
supervised fine-tuning, Chain of Thought (CoT) data is filtered using
ground-truth before fine-tuning the LLM, and (2) rule-based reinforcement
learning, which leverages PPO with ground-truth based rewards. We evaluate the
SoRFT-trained model on SWE-Bench Verified and SWE-Bench Lite, achieving
state-of-the-art (SOTA) performance among open-source models (e.g., resolve
21.4% issues on SWE-Bench Verified with SoRFT-Qwen-7B). The experimental
results demonstrate that SoRFT significantly enhances issue-resolving
performance, improves model generalization, and provides a cost-efficient
alternative to commercial models.
67c12de08cd49ca63e230b99
Zexiong Ma
claimed_verified
2025-02-28T09:28:35.503Z
654da66fb36f85a025bc24b6
/avatars/e5542856ab4bf1845e8f546b5f17cd99.svg
Zexiong Ma
user
mizersy
67c12de08cd49ca63e230b9a
Chao Peng
claimed_verified
2025-02-28T15:13:55.041Z
64425a502f4abae43fc0446c
/avatars/7448a8d024813d8a20e09c162a189304.svg
Chao Peng
user
pengchao
67c12de08cd49ca63e230b9b
Pengfei Gao
admin_assigned
2025-02-28T12:40:45.303Z
663f2dde4aeb9c177297fbd8
/avatars/ddc143481d2af893c9cdff1a33ccda28.svg
PengFei
user
PengFeiGao
67c12de08cd49ca63e230b9c
Xiangxin Meng
67c12de08cd49ca63e230b9d
Yanzhen Zou
67c12de08cd49ca63e230b9e
Bing Xie
SoRFT: Issue Resolving with Subtask-oriented Reinforced Fine-Tuning
Mainstream issue-resolving frameworks predominantly rely on commercial
models, leading to high costs and privacy concerns. Existing training
approaches for issue resolving struggle with poor generalization and fail to
fully leverage open-source development resources. We propose Subtask-oriented
Reinforced Fine-Tuning (SoRFT), a novel training approach to enhance the issue
resolving capability of LLMs. We decomposes issue resolving into structured
subtasks: file localization, function localization, line localization, and code
edit generation. SoRFT consists of two training stages: (1) rejection-sampled
supervised fine-tuning, Chain of Thought (CoT) data is filtered using
ground-truth before fine-tuning the LLM, and (2) rule-based reinforcement
learning, which leverages PPO with ground-truth based rewards. We evaluate the
SoRFT-trained model on SWE-Bench Verified and SWE-Bench Lite, achieving
state-of-the-art (SOTA) performance among open-source models (e.g., resolve
21.4% issues on SWE-Bench Verified with SoRFT-Qwen-7B). The experimental
results demonstrate that SoRFT significantly enhances issue-resolving
performance, improves model generalization, and provides a cost-efficient
alternative to commercial models.
Subtask-oriented Reinforced Fine-Tuning (SoRFT) enhances issue-resolving performance of LLMs through structured subtasks and a combination of supervised fine-tuning and rule-based reinforcement learning with PPO.
Subtask-oriented Reinforced Fine-Tuning SoRFT rejection-sampled supervised fine-tuning Chain of Thought data rule-based reinforcement learning Proximal Policy Optimization PPO SWE-Bench Verified SWE-Bench Lite