AlphaStar Unplugged: Large-Scale Offline Reinforcement Learning
StarCraft II is one of the most challenging simulated reinforcement learning
environments; it is partially observable, stochastic, multi-agent, and
mastering StarCraft II requires strategic planning over long time horizons with
real-time low-level execution. It also has an active professional competitive
scene. StarCraft II is uniquely suited for advancing offline RL algorithms,
both because of its challenging nature and because Blizzard has released a
massive dataset of millions of StarCraft II games played by human players. This
paper leverages that and establishes a benchmark, called AlphaStar Unplugged,
introducing unprecedented challenges for offline reinforcement learning. We
define a dataset (a subset of Blizzard's release), tools standardizing an API
for machine learning methods, and an evaluation protocol. We also present
baseline agents, including behavior cloning, offline variants of actor-critic
and MuZero. We improve the state of the art of agents using only offline data,
and we achieve 90% win rate against previously published AlphaStar behavior
cloning agent.
64d1a9f79617774ce42efbbf
Michaël Mathieu
64d1a9f79617774ce42efbc0
Sherjil Ozair
admin_assigned
2023-08-08T13:58:49.891Z
6311bda23a773344e068edc7
/avatars/4a7cdbdac92ad6bcb91d2f7d52f03613.svg
Sherjil
user
sherjilozair
64d1a9f79617774ce42efbc1
Srivatsan Srinivasan
64d1a9f79617774ce42efbc2
Caglar Gulcehre
claimed_verified
2023-08-09T00:43:02.684Z
64d28085db0b63b977a6c3f8
/avatars/994cc3ff7c18650fe2535e473d56b240.svg
Caglar Gulcehre
user
caglarg
64d1a9f79617774ce42efbc3
Shangtong Zhang
64d1a9f79617774ce42efbc4
Ray Jiang
64d1a9f79617774ce42efbc5
Tom Le Paine
64d1a9f79617774ce42efbc6
Richard Powell
64d1a9f79617774ce42efbc7
Konrad Żołna
64d1a9f79617774ce42efbc8
Julian Schrittwieser
64d1a9f79617774ce42efbc9
David Choi
64d1a9f79617774ce42efbca
Petko Georgiev
64d1a9f79617774ce42efbcb
Daniel Toyama
64d1a9f79617774ce42efbcc
Aja Huang
64d1a9f79617774ce42efbcd
Roman Ring
64d1a9f79617774ce42efbce
Igor Babuschkin
admin_assigned
2023-08-08T14:19:44.331Z
62d99bf239a1c86eec71bc41
/avatars/d2f3693eb76b95c7c7097cc6b5411fc4.svg
Igor Babuschkin
user
ibab
64d1a9f79617774ce42efbcf
Timo Ewalds
64d1a9f79617774ce42efbd0
Mahyar Bordbar
64d1a9f79617774ce42efbd1
Sarah Henderson
64d1a9f79617774ce42efbd2
Sergio Gómez Colmenarejo
64d1a9f79617774ce42efbd3
Aäron van den Oord
64d1a9f79617774ce42efbd4
Wojciech Marian Czarnecki
64d1a9f79617774ce42efbd5
Nando de Freitas
64d1a9f79617774ce42efbd6
Oriol Vinyals
AlphaStar Unplugged: Large-Scale Offline Reinforcement Learning
StarCraft II is one of the most challenging simulated reinforcement learning
environments; it is partially observable, stochastic, multi-agent, and
mastering StarCraft II requires strategic planning over long time horizons with
real-time low-level execution. It also has an active professional competitive
scene. StarCraft II is uniquely suited for advancing offline RL algorithms,
both because of its challenging nature and because Blizzard has released a
massive dataset of millions of StarCraft II games played by human players. This
paper leverages that and establishes a benchmark, called AlphaStar Unplugged,
introducing unprecedented challenges for offline reinforcement learning. We
define a dataset (a subset of Blizzard's release), tools standardizing an API
for machine learning methods, and an evaluation protocol. We also present
baseline agents, including behavior cloning, offline variants of actor-critic
and MuZero. We improve the state of the art of agents using only offline data,
and we achieve 90% win rate against previously published AlphaStar behavior
cloning agent.
A benchmark called AlphaStar Unplugged is established for offline reinforcement learning using a dataset from StarCraft II, featuring unprecedented challenges and achieving high win rates with offline data.
offline RL algorithms behavior cloning actor-critic MuZero AlphaStar Unplugged StarCraft II API evaluation protocol benchmark