title: Strategist: Learning Strategic Skills by LLMs via Bi-Level Tree Search
summary: In this paper, we propose a new method Strategist that utilizes LLMs to
  acquire new skills for playing multi-agent games through a self-improvement
  process. Our method gathers quality feedback through self-play simulations with
  Monte Carlo tree search and LLM-based reflection, which can then be used to
  learn high-level strategic skills such as how to evaluate states that guide the
  low-level execution.We showcase how our method can be used in both action
  planning and dialogue generation in the context of games, achieving good
  performance on both tasks. Specifically, we demonstrate that our method can
  help train agents with better performance than both traditional reinforcement
  learning-based approaches and other LLM-based skill learning approaches in
  games including the Game of Pure Strategy (GOPS) and The Resistance: Avalon.
paper_authors._id: 66c5780409ea098de4af0469
paper_authors.name: Jonathan Light
paper_authors._id: 66c5780409ea098de4af046a
paper_authors.name: Min Cai
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-22T19:51:15.899Z
paper_authors._id: 65af6d66ac291fd7fdd61bea
paper_authors.avatarUrl: /avatars/5c4db3f1ece631b28c52f57525dbcb92.svg
paper_authors.fullname: Min Cai
paper_authors.type: user
paper_authors.user: HenryCai1129
paper_authors._id: 66c5780409ea098de4af046b
paper_authors.name: Weiqin Chen
paper_authors._id: 66c5780409ea098de4af046c
paper_authors.name: Guanzhi Wang
paper_authors._id: 66c5780409ea098de4af046d
paper_authors.name: Xiusi Chen
paper_authors._id: 66c5780409ea098de4af046e
paper_authors.name: Wei Cheng
paper_authors._id: 66c5780409ea098de4af046f
paper_authors.name: Yisong Yue
paper_authors._id: 66c5780409ea098de4af0470
paper_authors.name: Ziniu Hu
paper_title: Strategist: Learning Strategic Skills by LLMs via Bi-Level Tree Search
paper_summary: In this paper, we propose a new method Strategist that utilizes LLMs to
  acquire new skills for playing multi-agent games through a self-improvement
  process. Our method gathers quality feedback through self-play simulations with
  Monte Carlo tree search and LLM-based reflection, which can then be used to
  learn high-level strategic skills such as how to evaluate states that guide the
  low-level execution.We showcase how our method can be used in both action
  planning and dialogue generation in the context of games, achieving good
  performance on both tasks. Specifically, we demonstrate that our method can
  help train agents with better performance than both traditional reinforcement
  learning-based approaches and other LLM-based skill learning approaches in
  games including the Game of Pure Strategy (GOPS) and The Resistance: Avalon.
paper_ai_keywords: LLMs self-improvement process self-play simulations Monte Carlo tree search LLM-based reflection high-level strategic skills low-level execution action planning dialogue generation reinforcement learning-based approaches Game of Pure Strategy (GOPS) The Resistance: Avalon