title: BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agents
summary: The massive successes of large language models (LLMs) encourage the emerging
  exploration of LLM-augmented Autonomous Agents (LAAs). An LAA is able to
  generate actions with its core LLM and interact with environments, which
  facilitates the ability to resolve complex tasks by conditioning on past
  interactions such as observations and actions. Since the investigation of LAA
  is still very recent, limited explorations are available. Therefore, we provide
  a comprehensive comparison of LAA in terms of both agent architectures and LLM
  backbones. Additionally, we propose a new strategy to orchestrate multiple LAAs
  such that each labor LAA focuses on one type of action, i.e. BOLAA,
  where a controller manages the communication among multiple agents. We conduct
  simulations on both decision-making and multi-step reasoning environments,
  which comprehensively justify the capacity of LAAs. Our performance results
  provide quantitative suggestions for designing LAA architectures and the
  optimal choice of LLMs, as well as the compatibility of both. We release our
  implementation code of LAAs to the public at
  https://github.com/salesforce/BOLAA.
paper_authors._id: 64d9ab56763279bb4d015564
paper_authors.name: Zhiwei Liu
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-02-27T00:56:31.072Z
paper_authors._id: 64470130bd9e0e0edf5227d7
paper_authors.avatarUrl: /avatars/89f118f880cce3d01658b123bbbf4402.svg
paper_authors.fullname: Zhiwei Liu
paper_authors.type: user
paper_authors.user: jimzhiwei
paper_authors._id: 64d9ab56763279bb4d015565
paper_authors.name: Weiran Yao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-08-15T00:33:44.312Z
paper_authors._id: 632cdea254e2c512c8f95b12
paper_authors.avatarUrl: /avatars/a6d06cdd75861ae7d589f1343d81a5c5.svg
paper_authors.fullname: Weiran Yao
paper_authors.type: user
paper_authors.user: weirayao
paper_authors._id: 64d9ab56763279bb4d015566
paper_authors.name: Jianguo Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-15T00:40:03.878Z
paper_authors._id: 630fcbd758d83e8f64d82777
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/630fcbd758d83e8f64d82777/0jwTa1MzeXZbDtJ-uXGo6.jpeg
paper_authors.fullname: Jianguo Zhang
paper_authors.type: user
paper_authors.user: jianguozhang
paper_authors._id: 64d9ab56763279bb4d015567
paper_authors.name: Le Xue
paper_authors._id: 64d9ab56763279bb4d015568
paper_authors.name: Shelby Heinecke
paper_authors._id: 64d9ab56763279bb4d015569
paper_authors.name: Rithesh Murthy
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-08-15T12:56:49.010Z
paper_authors._id: 60381555c58b41eeda0d6ccf
paper_authors.avatarUrl: /avatars/2c2c4b75dd0199478dc0fbc72f9bcadc.svg
paper_authors.fullname: rmurthy
paper_authors.type: user
paper_authors.user: rmurthy
paper_authors._id: 64d9ab56763279bb4d01556a
paper_authors.name: Yihao Feng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-15T00:40:49.549Z
paper_authors._id: 6354d31106d707b3324a3a5a
paper_authors.avatarUrl: /avatars/a662d176dfc4b809cd75aa726f24b74c.svg
paper_authors.fullname: Yihao Feng
paper_authors.type: user
paper_authors.user: yihaocs
paper_authors._id: 64d9ab56763279bb4d01556b
paper_authors.name: Zeyuan Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-15T00:41:14.659Z
paper_authors._id: 632405844624707127217677
paper_authors.avatarUrl: /avatars/af5deba9ec51814bc5ccfd04f68d76d3.svg
paper_authors.fullname: Zeyuan Chen
paper_authors.type: user
paper_authors.user: bradypus
paper_authors._id: 64d9ab56763279bb4d01556c
paper_authors.name: Juan Carlos Niebles
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-01T08:45:04.824Z
paper_authors._id: 65bb26b5d61b51a50805aba2
paper_authors.avatarUrl: /avatars/966c8469f55776948a7cf42d98accff3.svg
paper_authors.fullname: Juan Carlos Niebles
paper_authors.type: user
paper_authors.user: niebles
paper_authors._id: 64d9ab56763279bb4d01556d
paper_authors.name: Devansh Arpit
paper_authors._id: 64d9ab56763279bb4d01556e
paper_authors.name: Ran Xu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-19T22:15:10.196Z
paper_authors._id: 6465c4c863e7e09dd02e3e1b
paper_authors.avatarUrl: /avatars/200b029184d2616f98296a2c212f0785.svg
paper_authors.fullname: Ran Xu
paper_authors.type: user
paper_authors.user: xurantju
paper_authors._id: 64d9ab56763279bb4d01556f
paper_authors.name: Phil Mui
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-15T00:42:28.883Z
paper_authors._id: 63169c6618490c4b7a550ee2
paper_authors.avatarUrl: /avatars/c71726db2fdfea9a1c107eb16928517d.svg
paper_authors.fullname: Phil Mui
paper_authors.type: user
paper_authors.user: philmui
paper_authors._id: 64d9ab56763279bb4d015570
paper_authors.name: Huan Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-08-15T00:33:48.147Z
paper_authors._id: 64b96d0bea67d88ebec0f983
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64b96d0bea67d88ebec0f983/SnZwF6_ABRXhU64CoFoZm.jpeg
paper_authors.fullname: Huan Wang
paper_authors.type: user
paper_authors.user: joyousprince
paper_authors._id: 64d9ab56763279bb4d015571
paper_authors.name: Caiming Xiong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-15T00:42:45.151Z
paper_authors._id: 649dbcc4e0fff1ed099dc80a
paper_authors.avatarUrl: /avatars/c87c273ca628dbcddccbf1ee19b2ce33.svg
paper_authors.fullname: Caiming Xiong
paper_authors.type: user
paper_authors.user: cxiong
paper_authors._id: 64d9ab56763279bb4d015572
paper_authors.name: Silvio Savarese
paper_title: BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agents
paper_summary: The massive successes of large language models (LLMs) encourage the emerging
  exploration of LLM-augmented Autonomous Agents (LAAs). An LAA is able to
  generate actions with its core LLM and interact with environments, which
  facilitates the ability to resolve complex tasks by conditioning on past
  interactions such as observations and actions. Since the investigation of LAA
  is still very recent, limited explorations are available. Therefore, we provide
  a comprehensive comparison of LAA in terms of both agent architectures and LLM
  backbones. Additionally, we propose a new strategy to orchestrate multiple LAAs
  such that each labor LAA focuses on one type of action, i.e. BOLAA,
  where a controller manages the communication among multiple agents. We conduct
  simulations on both decision-making and multi-step reasoning environments,
  which comprehensively justify the capacity of LAAs. Our performance results
  provide quantitative suggestions for designing LAA architectures and the
  optimal choice of LLMs, as well as the compatibility of both. We release our
  implementation code of LAAs to the public at
  https://github.com/salesforce/BOLAA.
paper_ai_summary: A comprehensive comparison and new orchestration strategy for LLM-augmented Autonomous Agents (LAAs) demonstrate their effectiveness in decision-making and multi-step reasoning tasks.
paper_ai_keywords: large language models (LLMs) LLM-augmented Autonomous Agents (LAAs) agent architectures LLM backbones BOLAA decision-making multi-step reasoning