title: LLM-based Optimization of Compound AI Systems: A Survey
summary: In a compound AI system, components such as an LLM call, a retriever, a code
  interpreter, or tools are interconnected. The system's behavior is primarily
  driven by parameters such as instructions or tool definitions. Recent
  advancements enable end-to-end optimization of these parameters using an LLM.
  Notably, leveraging an LLM as an optimizer is particularly efficient because it
  avoids gradient computation and can generate complex code and instructions.
  This paper presents a survey of the principles and emerging trends in LLM-based
  optimization of compound AI systems. It covers archetypes of compound AI
  systems, approaches to LLM-based end-to-end optimization, and insights into
  future directions and broader impacts. Importantly, this survey uses concepts
  from program analysis to provide a unified view of how an LLM optimizer is
  prompted to optimize a compound AI system. The exhaustive list of paper is
  provided at
  https://github.com/linyuhongg/LLM-based-Optimization-of-Compound-AI-Systems.
paper_authors._id: 6719941a2d24f0c6bb1cf9db
paper_authors.name: Matthieu Lin
paper_authors._id: 6719941a2d24f0c6bb1cf9dc
paper_authors.name: Jenny Sheng
paper_authors._id: 6719941a2d24f0c6bb1cf9dd
paper_authors.name: Andrew Zhao
paper_authors._id: 6719941a2d24f0c6bb1cf9de
paper_authors.name: Shenzhi Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-24T07:15:34.990Z
paper_authors._id: 6486dde1f74857df3f1a5828
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6486dde1f74857df3f1a5828/FgE80CpalBO5qqArdfwxA.jpeg
paper_authors.fullname: Shenzhi Wang
paper_authors.type: user
paper_authors.user: shenzhi-wang
paper_authors._id: 6719941a2d24f0c6bb1cf9df
paper_authors.name: Yang Yue
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-05T07:58:48.839Z
paper_authors._id: 649d475111592b1a765ac1a3
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/649d475111592b1a765ac1a3/rjORJjErJq-mthghan08U.jpeg
paper_authors.fullname: Yang Yue
paper_authors.type: user
paper_authors.user: Yang130
paper_authors._id: 6719941a2d24f0c6bb1cf9e0
paper_authors.name: Yiran Wu
paper_authors._id: 6719941a2d24f0c6bb1cf9e1
paper_authors.name: Huan Liu
paper_authors._id: 6719941a2d24f0c6bb1cf9e2
paper_authors.name: Jun Liu
paper_authors._id: 6719941a2d24f0c6bb1cf9e3
paper_authors.name: Gao Huang
paper_authors._id: 6719941a2d24f0c6bb1cf9e4
paper_authors.name: Yong-Jin Liu
paper_title: LLM-based Optimization of Compound AI Systems: A Survey
paper_summary: In a compound AI system, components such as an LLM call, a retriever, a code
  interpreter, or tools are interconnected. The system's behavior is primarily
  driven by parameters such as instructions or tool definitions. Recent
  advancements enable end-to-end optimization of these parameters using an LLM.
  Notably, leveraging an LLM as an optimizer is particularly efficient because it
  avoids gradient computation and can generate complex code and instructions.
  This paper presents a survey of the principles and emerging trends in LLM-based
  optimization of compound AI systems. It covers archetypes of compound AI
  systems, approaches to LLM-based end-to-end optimization, and insights into
  future directions and broader impacts. Importantly, this survey uses concepts
  from program analysis to provide a unified view of how an LLM optimizer is
  prompted to optimize a compound AI system. The exhaustive list of paper is
  provided at
  https://github.com/linyuhongg/LLM-based-Optimization-of-Compound-AI-Systems.
paper_ai_keywords: LLM retriever code interpreter tools end-to-end optimization gradient computation compound AI systems LLM-based optimization archetypes approaches future directions broader impacts program analysis prompting