title: SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open
    Software Evolution
summary: The recent DeepSeek-R1 release has demonstrated the immense potential of
  reinforcement learning (RL) in enhancing the general reasoning capabilities of
  large language models (LLMs). While DeepSeek-R1 and other follow-up work
  primarily focus on applying RL to competitive coding and math problems, this
  paper introduces SWE-RL, the first approach to scale RL-based LLM reasoning for
  real-world software engineering. Leveraging a lightweight rule-based reward
  (e.g., the similarity score between ground-truth and LLM-generated solutions),
  SWE-RL enables LLMs to autonomously recover a developer's reasoning processes
  and solutions by learning from extensive open-source software evolution data --
  the record of a software's entire lifecycle, including its code snapshots, code
  changes, and events such as issues and pull requests. Trained on top of Llama
  3, our resulting reasoning model, Llama3-SWE-RL-70B, achieves a 41.0% solve
  rate on SWE-bench Verified -- a human-verified collection of real-world GitHub
  issues. To our knowledge, this is the best performance reported for
  medium-sized (<100B) LLMs to date, even comparable to leading proprietary LLMs
  like GPT-4o. Surprisingly, despite performing RL solely on software evolution
  data, Llama3-SWE-RL has even emerged with generalized reasoning skills. For
  example, it shows improved results on five out-of-domain tasks, namely,
  function coding, library use, code reasoning, mathematics, and general language
  understanding, whereas a supervised-finetuning baseline even leads to
  performance degradation on average. Overall, SWE-RL opens up a new direction to
  improve the reasoning capabilities of LLMs through reinforcement learning on
  massive software engineering data.
paper_authors._id: 67be845a8a5a80542314579f
paper_authors.name: Yuxiang Wei
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-26T08:50:44.837Z
paper_authors._id: 632a176259950c1d279d5ea7
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/632a176259950c1d279d5ea7/xsSGhBXalt9RaKzSKY8uk.jpeg
paper_authors.fullname: Yuxiang Wei
paper_authors.type: user
paper_authors.user: yuxiang630
paper_authors._id: 67be845a8a5a8054231457a0
paper_authors.name: Olivier Duchenne
paper_authors._id: 67be845a8a5a8054231457a1
paper_authors.name: Jade Copet
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-26T08:50:58.290Z
paper_authors._id: 6481e0ac50b759c75d5fdad0
paper_authors.avatarUrl: /avatars/49f08d989ca505ae01bce5578a94f6fe.svg
paper_authors.fullname: Jade Copet
paper_authors.type: user
paper_authors.user: JadeCopet
paper_authors._id: 67be845a8a5a8054231457a2
paper_authors.name: Quentin Carbonneaux
paper_authors._id: 67be845a8a5a8054231457a3
paper_authors.name: Lingming Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-26T08:51:10.640Z
paper_authors._id: 656f473c14fa8cfccd14559e
paper_authors.avatarUrl: /avatars/8f4fef3d835a7a11c2ab66dbf04f3424.svg
paper_authors.fullname: Lingming Zhang
paper_authors.type: user
paper_authors.user: lingming
paper_authors._id: 67be845a8a5a8054231457a4
paper_authors.name: Daniel Fried
paper_authors._id: 67be845a8a5a8054231457a5
paper_authors.name: Gabriel Synnaeve
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-26T08:51:21.641Z
paper_authors._id: 630eac7931970d1cd4fbacf2
paper_authors.avatarUrl: /avatars/b7ccbddfa745db854dc342be1327cd53.svg
paper_authors.fullname: Gabriel Synnaeve
paper_authors.type: user
paper_authors.user: gsynnaeve
paper_authors._id: 67be845a8a5a8054231457a6
paper_authors.name: Rishabh Singh
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-26T08:51:28.321Z
paper_authors._id: 6597e5a6420dcc68501a69e9
paper_authors.avatarUrl: /avatars/da48b13e07c367ecd5c891abfd6c3ded.svg
paper_authors.fullname: Rishabh Singh
paper_authors.type: user
paper_authors.user: RishabhSingh021
paper_authors._id: 67be845a8a5a8054231457a7
paper_authors.name: Sida I. Wang
paper_title: SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open
    Software Evolution
paper_summary: The recent DeepSeek-R1 release has demonstrated the immense potential of
  reinforcement learning (RL) in enhancing the general reasoning capabilities of
  large language models (LLMs). While DeepSeek-R1 and other follow-up work
  primarily focus on applying RL to competitive coding and math problems, this
  paper introduces SWE-RL, the first approach to scale RL-based LLM reasoning for
  real-world software engineering. Leveraging a lightweight rule-based reward
  (e.g., the similarity score between ground-truth and LLM-generated solutions),
  SWE-RL enables LLMs to autonomously recover a developer's reasoning processes
  and solutions by learning from extensive open-source software evolution data --
  the record of a software's entire lifecycle, including its code snapshots, code
  changes, and events such as issues and pull requests. Trained on top of Llama
  3, our resulting reasoning model, Llama3-SWE-RL-70B, achieves a 41.0% solve
  rate on SWE-bench Verified -- a human-verified collection of real-world GitHub
  issues. To our knowledge, this is the best performance reported for
  medium-sized (<100B) LLMs to date, even comparable to leading proprietary LLMs
  like GPT-4o. Surprisingly, despite performing RL solely on software evolution
  data, Llama3-SWE-RL has even emerged with generalized reasoning skills. For
  example, it shows improved results on five out-of-domain tasks, namely,
  function coding, library use, code reasoning, mathematics, and general language
  understanding, whereas a supervised-finetuning baseline even leads to
  performance degradation on average. Overall, SWE-RL opens up a new direction to
  improve the reasoning capabilities of LLMs through reinforcement learning on
  massive software engineering data.
paper_ai_summary: SWE-RL leverages reinforcement learning to enhance large language models' software engineering reasoning, achieving state-of-the-art performance and generalized reasoning skills by learning from extensive open-source software evolution data.
paper_ai_keywords: reinforcement learning large language models RL-based LLM reasoning lightweight rule-based reward open-source software evolution data SWE-bench Verified solve rate Llama 3 function coding library use code reasoning mathematics general language understanding