title: CodexGraph: Bridging Large Language Models and Code Repositories via
    Code Graph Databases
summary: Large Language Models (LLMs) excel in stand-alone code tasks like HumanEval
  and MBPP, but struggle with handling entire code repositories. This challenge
  has prompted research on enhancing LLM-codebase interaction at a repository
  scale. Current solutions rely on similarity-based retrieval or manual tools and
  APIs, each with notable drawbacks. Similarity-based retrieval often has low
  recall in complex tasks, while manual tools and APIs are typically
  task-specific and require expert knowledge, reducing their generalizability
  across diverse code tasks and real-world applications. To mitigate these
  limitations, we introduce \framework, a system that integrates LLM agents with
  graph database interfaces extracted from code repositories. By leveraging the
  structural properties of graph databases and the flexibility of the graph query
  language, \framework enables the LLM agent to construct and execute queries,
  allowing for precise, code structure-aware context retrieval and code
  navigation. We assess \framework using three benchmarks: CrossCodeEval,
  SWE-bench, and EvoCodeBench. Additionally, we develop five real-world coding
  applications. With a unified graph database schema, \framework demonstrates
  competitive performance and potential in both academic and real-world
  environments, showcasing its versatility and efficacy in software engineering.
  Our application demo:
  https://github.com/modelscope/modelscope-agent/tree/master/apps/codexgraph_agent.
paper_authors._id: 66b42c9b1c879ddf19205604
paper_authors.name: Xiangyan Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-08T10:50:54.107Z
paper_authors._id: 6486b09e8315b19342f0bf5e
paper_authors.avatarUrl: /avatars/bc5f22f231c884146d373fe1042d81bd.svg
paper_authors.fullname: Xiangyan Liu
paper_authors.type: user
paper_authors.user: xyliu6
paper_authors._id: 66b42c9b1c879ddf19205605
paper_authors.name: Bo Lan
paper_authors._id: 66b42c9b1c879ddf19205606
paper_authors.name: Zhiyuan Hu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-08T10:51:45.023Z
paper_authors._id: 65e9945f6bcbcae600b7e64f
paper_authors.avatarUrl: /avatars/8aa986a6c0e35c55d5d1461d1dc11ac3.svg
paper_authors.fullname: Zhiyuan Hu
paper_authors.type: user
paper_authors.user: zhiyhu
paper_authors._id: 66b42c9b1c879ddf19205607
paper_authors.name: Yang Liu
paper_authors._id: 66b42c9b1c879ddf19205608
paper_authors.name: Zhicheng Zhang
paper_authors._id: 66b42c9b1c879ddf19205609
paper_authors.name: Wenmeng Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-08T10:52:24.365Z
paper_authors._id: 623c6253389748c9f72ca287
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1654828369523-623c6253389748c9f72ca287.jpeg
paper_authors.fullname: wenmeng zhou
paper_authors.type: user
paper_authors.user: wenmengzhou
paper_authors._id: 66b42c9b1c879ddf1920560a
paper_authors.name: Fei Wang
paper_authors._id: 66b42c9b1c879ddf1920560b
paper_authors.name: Michael Shieh
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-08T10:53:41.871Z
paper_authors._id: 64d48e9ef2a3941669810f23
paper_authors.avatarUrl: /avatars/495e6785abc44d1facfcf385edd6a222.svg
paper_authors.fullname: Michael Shieh
paper_authors.type: user
paper_authors.user: Michael2008S
paper_title: CodexGraph: Bridging Large Language Models and Code Repositories via
    Code Graph Databases
paper_summary: Large Language Models (LLMs) excel in stand-alone code tasks like HumanEval
  and MBPP, but struggle with handling entire code repositories. This challenge
  has prompted research on enhancing LLM-codebase interaction at a repository
  scale. Current solutions rely on similarity-based retrieval or manual tools and
  APIs, each with notable drawbacks. Similarity-based retrieval often has low
  recall in complex tasks, while manual tools and APIs are typically
  task-specific and require expert knowledge, reducing their generalizability
  across diverse code tasks and real-world applications. To mitigate these
  limitations, we introduce \framework, a system that integrates LLM agents with
  graph database interfaces extracted from code repositories. By leveraging the
  structural properties of graph databases and the flexibility of the graph query
  language, \framework enables the LLM agent to construct and execute queries,
  allowing for precise, code structure-aware context retrieval and code
  navigation. We assess \framework using three benchmarks: CrossCodeEval,
  SWE-bench, and EvoCodeBench. Additionally, we develop five real-world coding
  applications. With a unified graph database schema, \framework demonstrates
  competitive performance and potential in both academic and real-world
  environments, showcasing its versatility and efficacy in software engineering.
  Our application demo:
  https://github.com/modelscope/modelscope-agent/tree/master/apps/codexgraph_agent.
paper_ai_summary: A system integrating LLMs with graph database interfaces for improved codebase interaction demonstrates competitive performance in real-world coding applications.
paper_ai_keywords: Large Language Models LLMs HumanEval MBPP similarity-based retrieval graph database interfaces graph query language CrossCodeEval SWE-bench EvoCodeBench code structure-aware context retrieval code navigation