title: Grandmaster-Level Chess Without Search
summary: The recent breakthrough successes in machine learning are mainly attributed
  to scale: namely large-scale attention-based architectures and datasets of
  unprecedented scale. This paper investigates the impact of training at scale
  for chess. Unlike traditional chess engines that rely on complex heuristics,
  explicit search, or a combination of both, we train a 270M parameter
  transformer model with supervised learning on a dataset of 10 million chess
  games. We annotate each board in the dataset with action-values provided by the
  powerful Stockfish 16 engine, leading to roughly 15 billion data points. Our
  largest model reaches a Lichess blitz Elo of 2895 against humans, and
  successfully solves a series of challenging chess puzzles, without any
  domain-specific tweaks or explicit search algorithms. We also show that our
  model outperforms AlphaZero's policy and value networks (without MCTS) and
  GPT-3.5-turbo-instruct. A systematic investigation of model and dataset size
  shows that strong chess performance only arises at sufficient scale. To
  validate our results, we perform an extensive series of ablations of design
  choices and hyperparameters.
paper_authors._id: 65c4687bc3e219990818da3c
paper_authors.name: Anian Ruoss
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-09T08:05:26.326Z
paper_authors._id: 65c51758522ee3a4edd917d0
paper_authors.avatarUrl: /avatars/2ea3b64b5eeb34e2bfe821e4210f1b3e.svg
paper_authors.fullname: Anian Ruoss
paper_authors.type: user
paper_authors.user: anianruoss
paper_authors._id: 65c4687bc3e219990818da3d
paper_authors.name: Grégoire Delétang
paper_authors._id: 65c4687bc3e219990818da3e
paper_authors.name: Sourabh Medapati
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-08T09:51:24.772Z
paper_authors._id: 648888b165e25cc87b6ab206
paper_authors.avatarUrl: /avatars/c972e4ff17a446c4d2f803fe9315ce12.svg
paper_authors.fullname: Sourabh Medapati
paper_authors.type: user
paper_authors.user: smedapati
paper_authors._id: 65c4687bc3e219990818da3f
paper_authors.name: Jordi Grau-Moya
paper_authors._id: 65c4687bc3e219990818da40
paper_authors.name: Li Kevin Wenliang
paper_authors._id: 65c4687bc3e219990818da41
paper_authors.name: Elliot Catt
paper_authors._id: 65c4687bc3e219990818da42
paper_authors.name: John Reid
paper_authors._id: 65c4687bc3e219990818da43
paper_authors.name: Tim Genewein
paper_title: Grandmaster-Level Chess Without Search
paper_summary: The recent breakthrough successes in machine learning are mainly attributed
  to scale: namely large-scale attention-based architectures and datasets of
  unprecedented scale. This paper investigates the impact of training at scale
  for chess. Unlike traditional chess engines that rely on complex heuristics,
  explicit search, or a combination of both, we train a 270M parameter
  transformer model with supervised learning on a dataset of 10 million chess
  games. We annotate each board in the dataset with action-values provided by the
  powerful Stockfish 16 engine, leading to roughly 15 billion data points. Our
  largest model reaches a Lichess blitz Elo of 2895 against humans, and
  successfully solves a series of challenging chess puzzles, without any
  domain-specific tweaks or explicit search algorithms. We also show that our
  model outperforms AlphaZero's policy and value networks (without MCTS) and
  GPT-3.5-turbo-instruct. A systematic investigation of model and dataset size
  shows that strong chess performance only arises at sufficient scale. To
  validate our results, we perform an extensive series of ablations of design
  choices and hyperparameters.
paper_ai_keywords: attention-based architectures large parameter transformer model supervised learning dataset action-values Stockfish 16 engine Lblitz Elo challenged chess puzzles policy and value networks MCTS GPT-3.5-turbo-instruct