title: PokéChamp: an Expert-level Minimax Language Agent
summary: We introduce Pok\'eChamp, a minimax agent powered by Large Language Models
  (LLMs) for Pok\'emon battles. Built on a general framework for two-player
  competitive games, Pok\'eChamp leverages the generalist capabilities of LLMs to
  enhance minimax tree search. Specifically, LLMs replace three key modules: (1)
  player action sampling, (2) opponent modeling, and (3) value function
  estimation, enabling the agent to effectively utilize gameplay history and
  human knowledge to reduce the search space and address partial observability.
  Notably, our framework requires no additional LLM training. We evaluate
  Pok\'eChamp in the popular Gen 9 OU format. When powered by GPT-4o, it achieves
  a win rate of 76% against the best existing LLM-based bot and 84% against the
  strongest rule-based bot, demonstrating its superior performance. Even with an
  open-source 8-billion-parameter Llama 3.1 model, Pok\'eChamp consistently
  outperforms the previous best LLM-based bot, Pok\'ellmon powered by GPT-4o,
  with a 64% win rate. Pok\'eChamp attains a projected Elo of 1300-1500 on the
  Pok\'emon Showdown online ladder, placing it among the top 30%-10% of human
  players. In addition, this work compiles the largest real-player Pok\'emon
  battle dataset, featuring over 3 million games, including more than 500k
  high-Elo matches. Based on this dataset, we establish a series of battle
  benchmarks and puzzles to evaluate specific battling skills. We further provide
  key updates to the local game engine. We hope this work fosters further
  research that leverage Pok\'emon battle as benchmark to integrate LLM
  technologies with game-theoretic algorithms addressing general multiagent
  problems. Videos, code, and dataset available at
  https://sites.google.com/view/pokechamp-llm.
paper_authors._id: 67ca7bcc06501013d727a5d7
paper_authors.name: Seth Karten
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-03-07T06:07:26.564Z
paper_authors._id: 6658e1c8ce1b2838885b2d7f
paper_authors.avatarUrl: /avatars/8623555f14b62f40fd372da20cb59ccc.svg
paper_authors.fullname: Seth Karten
paper_authors.type: user
paper_authors.user: milkkarten
paper_authors._id: 67ca7bcc06501013d727a5d8
paper_authors.name: Andy Luu Nguyen
paper_authors._id: 67ca7bcc06501013d727a5d9
paper_authors.name: Chi Jin
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-03-07T04:53:33.942Z
paper_authors._id: 66749c510974bbc971139f6a
paper_authors.avatarUrl: /avatars/bfab9d8d8bc589bb9bd49925b76e04a4.svg
paper_authors.fullname: Chi Jin
paper_authors.type: user
paper_authors.user: chijin
paper_title: PokéChamp: an Expert-level Minimax Language Agent
paper_summary: We introduce Pok\'eChamp, a minimax agent powered by Large Language Models
  (LLMs) for Pok\'emon battles. Built on a general framework for two-player
  competitive games, Pok\'eChamp leverages the generalist capabilities of LLMs to
  enhance minimax tree search. Specifically, LLMs replace three key modules: (1)
  player action sampling, (2) opponent modeling, and (3) value function
  estimation, enabling the agent to effectively utilize gameplay history and
  human knowledge to reduce the search space and address partial observability.
  Notably, our framework requires no additional LLM training. We evaluate
  Pok\'eChamp in the popular Gen 9 OU format. When powered by GPT-4o, it achieves
  a win rate of 76% against the best existing LLM-based bot and 84% against the
  strongest rule-based bot, demonstrating its superior performance. Even with an
  open-source 8-billion-parameter Llama 3.1 model, Pok\'eChamp consistently
  outperforms the previous best LLM-based bot, Pok\'ellmon powered by GPT-4o,
  with a 64% win rate. Pok\'eChamp attains a projected Elo of 1300-1500 on the
  Pok\'emon Showdown online ladder, placing it among the top 30%-10% of human
  players. In addition, this work compiles the largest real-player Pok\'emon
  battle dataset, featuring over 3 million games, including more than 500k
  high-Elo matches. Based on this dataset, we establish a series of battle
  benchmarks and puzzles to evaluate specific battling skills. We further provide
  key updates to the local game engine. We hope this work fosters further
  research that leverage Pok\'emon battle as benchmark to integrate LLM
  technologies with game-theoretic algorithms addressing general multiagent
  problems. Videos, code, and dataset available at
  https://sites.google.com/view/pokechamp-llm.
paper_ai_summary: Pok'eChamp, a minimax agent using LLMs for Pokémon battles, demonstrates superior performance against existing bots through enhanced action sampling, opponent modeling, and value function estimation.
paper_ai_keywords: minimax agent Large Language Models (LLMs) player action sampling opponent modeling value function estimation gameplay history partial observability GPT-4o Llama 3.1 Elo Pokémon Showdown battle dataset battle benchmarks puzzle evaluation game-theoretic algorithms multiagent problems