title: Diversity Empowers Intelligence: Integrating Expertise of Software
    Engineering Agents
summary: Large language model (LLM) agents have shown great potential in solving
  real-world software engineering (SWE) problems. The most advanced open-source
  SWE agent can resolve over 27% of real GitHub issues in SWE-Bench Lite.
  However, these sophisticated agent frameworks exhibit varying strengths,
  excelling in certain tasks while underperforming in others. To fully harness
  the diversity of these agents, we propose DEI (Diversity Empowered
  Intelligence), a framework that leverages their unique expertise. DEI functions
  as a meta-module atop existing SWE agent frameworks, managing agent collectives
  for enhanced problem-solving. Experimental results show that a DEI-guided
  committee of agents is able to surpass the best individual agent's performance
  by a large margin. For instance, a group of open-source SWE agents, with a
  maximum individual resolve rate of 27.3% on SWE-Bench Lite, can achieve a 34.3%
  resolve rate with DEI, making a 25% improvement and beating most closed-source
  solutions. Our best-performing group excels with a 55% resolve rate, securing
  the highest ranking on SWE-Bench Lite. Our findings contribute to the growing
  body of research on collaborative AI systems and their potential to solve
  complex software engineering challenges.
paper_authors._id: 66bcd64b85c8bcb5f2681504
paper_authors.name: Kexun Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-15T10:11:03.448Z
paper_authors._id: 62ee423b4bebb4ab55c674b1
paper_authors.avatarUrl: /avatars/ce2797937e8225937fc84d6847d50077.svg
paper_authors.fullname: Kexun Zhang
paper_authors.type: user
paper_authors.user: k1z
paper_authors._id: 66bcd64b85c8bcb5f2681505
paper_authors.name: Weiran Yao
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-08-14T16:07:39.897Z
paper_authors._id: 661573234c2f29635e93bb71
paper_authors.avatarUrl: /avatars/fba95e382454485766b6349d6281b715.svg
paper_authors.fullname: Weiran Yao
paper_authors.type: user
paper_authors.user: weiranyao
paper_authors._id: 66bcd64b85c8bcb5f2681506
paper_authors.name: Zuxin Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-15T10:09:35.758Z
paper_authors._id: 638e971f2cc490759fe9210e
paper_authors.avatarUrl: /avatars/9578d0badc084d340c028a59e571abd3.svg
paper_authors.fullname: Liu
paper_authors.type: user
paper_authors.user: Zuxin
paper_authors._id: 66bcd64b85c8bcb5f2681507
paper_authors.name: Yihao Feng
paper_authors._id: 66bcd64b85c8bcb5f2681508
paper_authors.name: Zhiwei Liu
paper_authors._id: 66bcd64b85c8bcb5f2681509
paper_authors.name: Rithesh Murthy
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-15T10:09:33.543Z
paper_authors._id: 60381555c58b41eeda0d6ccf
paper_authors.avatarUrl: /avatars/2c2c4b75dd0199478dc0fbc72f9bcadc.svg
paper_authors.fullname: rmurthy
paper_authors.type: user
paper_authors.user: rmurthy
paper_authors._id: 66bcd64b85c8bcb5f268150a
paper_authors.name: Tian Lan
paper_authors._id: 66bcd64b85c8bcb5f268150b
paper_authors.name: Lei Li
paper_authors._id: 66bcd64b85c8bcb5f268150c
paper_authors.name: Renze Lou
paper_authors._id: 66bcd64b85c8bcb5f268150d
paper_authors.name: Jiacheng Xu
paper_authors._id: 66bcd64b85c8bcb5f268150e
paper_authors.name: Bo Pang
paper_authors._id: 66bcd64b85c8bcb5f268150f
paper_authors.name: Yingbo Zhou
paper_authors._id: 66bcd64b85c8bcb5f2681510
paper_authors.name: Shelby Heinecke
paper_authors._id: 66bcd64b85c8bcb5f2681511
paper_authors.name: Silvio Savarese
paper_authors._id: 66bcd64b85c8bcb5f2681512
paper_authors.name: Huan Wang
paper_authors._id: 66bcd64b85c8bcb5f2681513
paper_authors.name: Caiming Xiong
paper_title: Diversity Empowers Intelligence: Integrating Expertise of Software
    Engineering Agents
paper_summary: Large language model (LLM) agents have shown great potential in solving
  real-world software engineering (SWE) problems. The most advanced open-source
  SWE agent can resolve over 27% of real GitHub issues in SWE-Bench Lite.
  However, these sophisticated agent frameworks exhibit varying strengths,
  excelling in certain tasks while underperforming in others. To fully harness
  the diversity of these agents, we propose DEI (Diversity Empowered
  Intelligence), a framework that leverages their unique expertise. DEI functions
  as a meta-module atop existing SWE agent frameworks, managing agent collectives
  for enhanced problem-solving. Experimental results show that a DEI-guided
  committee of agents is able to surpass the best individual agent's performance
  by a large margin. For instance, a group of open-source SWE agents, with a
  maximum individual resolve rate of 27.3% on SWE-Bench Lite, can achieve a 34.3%
  resolve rate with DEI, making a 25% improvement and beating most closed-source
  solutions. Our best-performing group excels with a 55% resolve rate, securing
  the highest ranking on SWE-Bench Lite. Our findings contribute to the growing
  body of research on collaborative AI systems and their potential to solve
  complex software engineering challenges.
paper_ai_summary: A DEI framework enhances software engineering problem-solving by leveraging a collective of LLM agents, achieving higher resolve rates compared to individual agents and closed-source solutions.
paper_ai_keywords: LLM agents software engineering SWE agent frameworks DEI (Diversity Empowered Intelligence) meta-module agent collectives SWE-Bench Lite resolve rate collaborative AI systems