PokéChamp: an Expert-level Minimax Language Agent
We introduce Pok\'eChamp, a minimax agent powered by Large Language Models
(LLMs) for Pok\'emon battles. Built on a general framework for two-player
competitive games, Pok\'eChamp leverages the generalist capabilities of LLMs to
enhance minimax tree search. Specifically, LLMs replace three key modules: (1)
player action sampling, (2) opponent modeling, and (3) value function
estimation, enabling the agent to effectively utilize gameplay history and
human knowledge to reduce the search space and address partial observability.
Notably, our framework requires no additional LLM training. We evaluate
Pok\'eChamp in the popular Gen 9 OU format. When powered by GPT-4o, it achieves
a win rate of 76% against the best existing LLM-based bot and 84% against the
strongest rule-based bot, demonstrating its superior performance. Even with an
open-source 8-billion-parameter Llama 3.1 model, Pok\'eChamp consistently
outperforms the previous best LLM-based bot, Pok\'ellmon powered by GPT-4o,
with a 64% win rate. Pok\'eChamp attains a projected Elo of 1300-1500 on the
Pok\'emon Showdown online ladder, placing it among the top 30%-10% of human
players. In addition, this work compiles the largest real-player Pok\'emon
battle dataset, featuring over 3 million games, including more than 500k
high-Elo matches. Based on this dataset, we establish a series of battle
benchmarks and puzzles to evaluate specific battling skills. We further provide
key updates to the local game engine. We hope this work fosters further
research that leverage Pok\'emon battle as benchmark to integrate LLM
technologies with game-theoretic algorithms addressing general multiagent
problems. Videos, code, and dataset available at
https://sites.google.com/view/pokechamp-llm.
67ca7bcc06501013d727a5d7
Seth Karten
extracted_confirmed
2025-03-07T06:07:26.564Z
6658e1c8ce1b2838885b2d7f
/avatars/8623555f14b62f40fd372da20cb59ccc.svg
Seth Karten
user
milkkarten
67ca7bcc06501013d727a5d8
Andy Luu Nguyen
67ca7bcc06501013d727a5d9
Chi Jin
extracted_pending
2025-03-07T04:53:33.942Z
66749c510974bbc971139f6a
/avatars/bfab9d8d8bc589bb9bd49925b76e04a4.svg
Chi Jin
user
chijin
PokéChamp: an Expert-level Minimax Language Agent
We introduce Pok\'eChamp, a minimax agent powered by Large Language Models
(LLMs) for Pok\'emon battles. Built on a general framework for two-player
competitive games, Pok\'eChamp leverages the generalist capabilities of LLMs to
enhance minimax tree search. Specifically, LLMs replace three key modules: (1)
player action sampling, (2) opponent modeling, and (3) value function
estimation, enabling the agent to effectively utilize gameplay history and
human knowledge to reduce the search space and address partial observability.
Notably, our framework requires no additional LLM training. We evaluate
Pok\'eChamp in the popular Gen 9 OU format. When powered by GPT-4o, it achieves
a win rate of 76% against the best existing LLM-based bot and 84% against the
strongest rule-based bot, demonstrating its superior performance. Even with an
open-source 8-billion-parameter Llama 3.1 model, Pok\'eChamp consistently
outperforms the previous best LLM-based bot, Pok\'ellmon powered by GPT-4o,
with a 64% win rate. Pok\'eChamp attains a projected Elo of 1300-1500 on the
Pok\'emon Showdown online ladder, placing it among the top 30%-10% of human
players. In addition, this work compiles the largest real-player Pok\'emon
battle dataset, featuring over 3 million games, including more than 500k
high-Elo matches. Based on this dataset, we establish a series of battle
benchmarks and puzzles to evaluate specific battling skills. We further provide
key updates to the local game engine. We hope this work fosters further
research that leverage Pok\'emon battle as benchmark to integrate LLM
technologies with game-theoretic algorithms addressing general multiagent
problems. Videos, code, and dataset available at
https://sites.google.com/view/pokechamp-llm.
Pok'eChamp, a minimax agent using LLMs for Pokémon battles, demonstrates superior performance against existing bots through enhanced action sampling, opponent modeling, and value function estimation.
minimax agent Large Language Models (LLMs) player action sampling opponent modeling value function estimation gameplay history partial observability GPT-4o Llama 3.1 Elo Pokémon Showdown battle dataset battle benchmarks puzzle evaluation game-theoretic algorithms multiagent problems