title: Training Software Engineering Agents and Verifiers with SWE-Gym
summary: We present SWE-Gym, the first environment for training real-world software
  engineering (SWE) agents. SWE-Gym contains 2,438 real-world Python task
  instances, each comprising a codebase with an executable runtime environment,
  unit tests, and a task specified in natural language. We use SWE-Gym to train
  language model based SWE agents , achieving up to 19% absolute gains in resolve
  rate on the popular SWE-Bench Verified and Lite test sets. We also experiment
  with inference-time scaling through verifiers trained on agent trajectories
  sampled from SWE-Gym. When combined with our fine-tuned SWE agents, we achieve
  32.0% and 26.0% on SWE-Bench Verified and Lite, respectively, reflecting a new
  state-of-the-art for open-weight SWE agents. To facilitate further research, we
  publicly release SWE-Gym, models, and agent trajectories.
paper_authors._id: 67735961702f3c89046839f4
paper_authors.name: Jiayi Pan
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-12-31T05:07:53.061Z
paper_authors._id: 61568f37272f2d87a99ba884
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61568f37272f2d87a99ba884/lgvkl5f0rEyiQRVU5FE32.png
paper_authors.fullname: Jiayi Pan
paper_authors.type: user
paper_authors.user: Jiayi-Pan
paper_authors._id: 67735961702f3c89046839f5
paper_authors.name: Xingyao Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-01T20:13:13.208Z
paper_authors._id: 62c63f31031996c36c848504
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62c63f31031996c36c848504/xZvH6jnozuFLXbtRSU-h1.jpeg
paper_authors.fullname: Xingyao Wang
paper_authors.type: user
paper_authors.user: xingyaoww
paper_authors._id: 67735961702f3c89046839f6
paper_authors.name: Graham Neubig
paper_authors._id: 67735961702f3c89046839f7
paper_authors.name: Navdeep Jaitly
paper_authors._id: 67735961702f3c89046839f8
paper_authors.name: Heng Ji
paper_authors._id: 67735961702f3c89046839f9
paper_authors.name: Alane Suhr
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-01-02T00:34:15.047Z
paper_authors._id: 6611e6e1188ff298b0dd0b79
paper_authors.avatarUrl: /avatars/3a495283955ec9e06e1829c7eb2cd9a4.svg
paper_authors.fullname: Alane Suhr
paper_authors.type: user
paper_authors.user: alsuhr
paper_authors._id: 67735961702f3c89046839fa
paper_authors.name: Yizhe Zhang
paper_title: Training Software Engineering Agents and Verifiers with SWE-Gym
paper_summary: We present SWE-Gym, the first environment for training real-world software
  engineering (SWE) agents. SWE-Gym contains 2,438 real-world Python task
  instances, each comprising a codebase with an executable runtime environment,
  unit tests, and a task specified in natural language. We use SWE-Gym to train
  language model based SWE agents , achieving up to 19% absolute gains in resolve
  rate on the popular SWE-Bench Verified and Lite test sets. We also experiment
  with inference-time scaling through verifiers trained on agent trajectories
  sampled from SWE-Gym. When combined with our fine-tuned SWE agents, we achieve
  32.0% and 26.0% on SWE-Bench Verified and Lite, respectively, reflecting a new
  state-of-the-art for open-weight SWE agents. To facilitate further research, we
  publicly release SWE-Gym, models, and agent trajectories.