title: SWE-smith: Scaling Data for Software Engineering Agents
summary: Despite recent progress in Language Models (LMs) for software engineering,
  collecting training data remains a significant pain point. Existing datasets
  are small, with at most 1,000s of training instances from 11 or fewer GitHub
  repositories. The procedures to curate such datasets are often complex,
  necessitating hundreds of hours of human labor; companion execution
  environments also take up several terabytes of storage, severely limiting their
  scalability and usability. To address this pain point, we introduce SWE-smith,
  a novel pipeline for generating software engineering training data at scale.
  Given any Python codebase, SWE-smith constructs a corresponding execution
  environment, then automatically synthesizes 100s to 1,000s of task instances
  that break existing test(s) in the codebase. Using SWE-smith, we create a
  dataset of 50k instances sourced from 128 GitHub repositories, an order of
  magnitude larger than all previous works. We train SWE-agent-LM-32B, achieving
  40.2% Pass@1 resolve rate on the SWE-bench Verified benchmark, state of the art
  among open source models. We open source SWE-smith (collection procedure, task
  instances, trajectories, models) to lower the barrier of entry for research in
  LM systems for automated software engineering. All assets available at
  https://swesmith.com.
paper_authors._id: 681b80bd2552f1531d8be2b7
paper_authors.name: John Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-08T08:59:16.662Z
paper_authors._id: 677c2e5032582d07139f8af1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/ZxdbXnPAFq_O20FAZuD4v.png
paper_authors.fullname: John
paper_authors.type: user
paper_authors.user: john-b-yang
paper_authors._id: 681b80bd2552f1531d8be2b8
paper_authors.name: Kilian Leret
paper_authors._id: 681b80bd2552f1531d8be2b9
paper_authors.name: Carlos E. Jimenez
paper_authors._id: 681b80bd2552f1531d8be2ba
paper_authors.name: Alexander Wettig
paper_authors._id: 681b80bd2552f1531d8be2bb
paper_authors.name: Kabir Khandpur
paper_authors._id: 681b80bd2552f1531d8be2bc
paper_authors.name: Yanzhe Zhang
paper_authors._id: 681b80bd2552f1531d8be2bd
paper_authors.name: Binyuan Hui
paper_authors._id: 681b80bd2552f1531d8be2be
paper_authors.name: Ofir Press
paper_authors._id: 681b80bd2552f1531d8be2bf
paper_authors.name: Ludwig Schmidt
paper_authors._id: 681b80bd2552f1531d8be2c0
paper_authors.name: Diyi Yang
paper_title: SWE-smith: Scaling Data for Software Engineering Agents
paper_summary: Despite recent progress in Language Models (LMs) for software engineering,
  collecting training data remains a significant pain point. Existing datasets
  are small, with at most 1,000s of training instances from 11 or fewer GitHub
  repositories. The procedures to curate such datasets are often complex,
  necessitating hundreds of hours of human labor; companion execution
  environments also take up several terabytes of storage, severely limiting their
  scalability and usability. To address this pain point, we introduce SWE-smith,
  a novel pipeline for generating software engineering training data at scale.
  Given any Python codebase, SWE-smith constructs a corresponding execution
  environment, then automatically synthesizes 100s to 1,000s of task instances
  that break existing test(s) in the codebase. Using SWE-smith, we create a
  dataset of 50k instances sourced from 128 GitHub repositories, an order of
  magnitude larger than all previous works. We train SWE-agent-LM-32B, achieving
  40.2% Pass@1 resolve rate on the SWE-bench Verified benchmark, state of the art
  among open source models. We open source SWE-smith (collection procedure, task
  instances, trajectories, models) to lower the barrier of entry for research in
  LM systems for automated software engineering. All assets available at
  https://swesmith.com.
paper_ai_summary: SWE-smith automates the generation of large-scale software engineering training data and improves the performance of language models on automated software engineering tasks.
paper_ai_keywords: Language Models software engineering training data GitHub repositories execution environment task instances SWE-smith SWE-agent-LM-32B SWE-bench Verified benchmark automated software engineering