Tool-integrated Reinforcement Learning for Repo Deep Search
Issue localization, the process of identifying code locations that need
modification to resolve software issues, is a critical yet challenging task in
software development. The semantic gap between natural language issue
descriptions and faulty code requires complex multi-hop reasoning through code
dependencies. Existing LLM-based agents attempt to address this by integrating
repository retrieval tools. However, this transforms issue localization into a
demanding task we call Repo Deep Search, which requires the LLM to effectively
utilize various repository retrieval tools throughout a multi-step reasoning
and navigation process. To tackle this challenge, we present ToolTrain, a
two-stage tool-integrated training framework combining rejection-sampled
supervised fine-tuning and tool-integrated reinforcement learning to enhance
LLMs' ability to use retrieval tools for issue localization. Experimental
results show that ToolTrain-trained models achieve state-of-the-art
performance, with our 32B model even surpassing Claude-3.7 on function-level
localization. The results also show that improved localization performance
translates to better end-to-end issue resolution performance. This further
demonstrates that training for issue localization is a viable and effective
strategy for improving automated software development.
6892c65c8da45ffb0a2b245e
Zexiong Ma
6892c65c8da45ffb0a2b245f
Chao Peng
6892c65c8da45ffb0a2b2460
Qunhong Zeng
6892c65c8da45ffb0a2b2461
Pengfei Gao
6892c65c8da45ffb0a2b2462
Yanzhen Zou
6892c65c8da45ffb0a2b2463
Bing Xie
Tool-integrated Reinforcement Learning for Repo Deep Search
Issue localization, the process of identifying code locations that need
modification to resolve software issues, is a critical yet challenging task in
software development. The semantic gap between natural language issue
descriptions and faulty code requires complex multi-hop reasoning through code
dependencies. Existing LLM-based agents attempt to address this by integrating
repository retrieval tools. However, this transforms issue localization into a
demanding task we call Repo Deep Search, which requires the LLM to effectively
utilize various repository retrieval tools throughout a multi-step reasoning
and navigation process. To tackle this challenge, we present ToolTrain, a
two-stage tool-integrated training framework combining rejection-sampled
supervised fine-tuning and tool-integrated reinforcement learning to enhance
LLMs' ability to use retrieval tools for issue localization. Experimental
results show that ToolTrain-trained models achieve state-of-the-art
performance, with our 32B model even surpassing Claude-3.7 on function-level
localization. The results also show that improved localization performance
translates to better end-to-end issue resolution performance. This further
demonstrates that training for issue localization is a viable and effective
strategy for improving automated software development.
ToolTrain, a two-stage training framework combining supervised fine-tuning and reinforcement learning, enhances LLMs for issue localization by integrating repository retrieval tools, achieving state-of-the-art performance.
LLM-based agents Repo Deep Search rejection-sampled supervised fine-tuning tool-integrated reinforcement learning function-level localization Claude-3.7 automated software development