ACC-Collab: An Actor-Critic Approach to Multi-Agent LLM Collaboration
Andrew Estornell Jean-Francois Ton Yuanshun Yao Yang Liu
Large language models (LLMs) have demonstrated a remarkable ability to serve as general-purpose tools for various language-based tasks. 
  Recent works have demonstrated that the efficacy of such models can be improved through iterative dialog between multiple models.
  While these 
  paradigms show promise 
  in
  improving model efficacy, most works in this area treat collaboration as an emergent behavior, rather than a learned behavior. 
  In doing so, current multi-agent frameworks rely on collaborative behaviors to have been sufficiently trained into off-the-shelf models. 
  To address this limitation, we propose ACC-Collab, an **A**ctor-**C**riti**c** based learning framework to produce a two-agent team (an actor-agent and a critic-agent) specialized in collaboration.
  We demonstrate that ACC-Collab outperforms SotA multi-agent techniques on a wide array of benchmarks.
Multi-Agent Collaboration, LLM Agents, Preference Optimization
We train a 2-agent LLM team (one actor-agent and one critic-agent) to collaboratively solve problems.