Answers.Body: <p>For board games, you may not need an advanced computer to achieve very good results. TPU and high-end GPUs allow to go one step further of course.</p>
  <p>But there is an existing implementation on github alpha-zero-general (not from me). By adding some optimizations tricks from scientific articles, I managed to train alphazero for several board games (Splendor, Santorini, Machi Koro, The Little Prince Make Me A Planet) with my i5 CPU. You may not need GPU as most of the time is spent in self-playing more than training, meaning that computer needs to do lots of network inferences more than gradient. After about a day of training, the training seemed to reach its asymptote. As a result, the computer was winning over me 100% on each game.</p>
  <p>I don't want to promote my self github, just telling OP that is feasible.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-23T07:47:17.850Z
Answers.LastActivityDate: 2022-07-23T07:47:17.850Z
Answers.UserDisplayName: cestpasfaux
Answers.Body: <p>The vast majority of neural networks are now trained on graphics processing units (GPUs) or specialised accelerator hardware such as <a href="https://en.wikipedia.org/wiki/Tensor_Processing_Unit" rel="nofollow noreferrer">tensor processing units</a> (TPUs).</p>
  <p>In <a href="https://arxiv.org/pdf/1712.01815.pdf" rel="nofollow noreferrer">Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm</a>, Silver et al. say that the training process involved 5,000 first-generation TPUs generating self-play games and 64 second-generation TPUs for training. This is certainly far beyond what any practical gaming computer is likely to achieve, as you'll likely only have one GPU, and that might not even rival a single TPU. Training on the CPU will be substantially slower again than either a GPU or TPU. Training would be orders of magnitude slower; you might find <a href="https://arxiv.org/pdf/1907.10701.pdf" rel="nofollow noreferrer">these benchmarks</a> by Wang et al. of interest.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-02-15T19:23:12.457Z
Answers.Text: Not arguing with any part of this answer, and I may add that I think gcp (creator of Leela zero) estimated something like 10000 years in single GPU (estimated 3 years ago). However this training time depends on the game complexity a lot and also if the game has no professional players then a single GPU may well be the best in the world after a few months if not a few weeks or even days.
Answers.UserDisplayName: davik
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-02-15T17:28:49.367Z
Answers.LastActivityDate: 2021-02-15T17:28:49.367Z
Answers.UserDisplayName: htl
Body: <p>Is it practical/affordable to train an AlphaZero/MuZero engine using a residential gaming PC, or would it take thousands of years of training for the AI to learn enough to challenge humans?</p>
  <p>I'm having trouble wrapping my head around how much computing power '4 hours of Google DeepMind training' equates to my residential computer running 24/7 trying to build a trained AI.</p>
  <p>Basically, are AlphaZero or MuZero practical for indie board games that want a state of the art AI, or is it too expensive to train?</p>
  
Tags: <training><game-ai><alphazero><muzero>
Title: Is it practical to train AlphaZero or MuZero (for indie games) on a personal computer?
UserDisplayName: Luke W
ThreadText: # Is it practical to train AlphaZero or MuZero (for indie games) on a personal computer?
  
  (Asked by: username_0 on 2021-02-15)
  
  Is it practical/affordable to train an AlphaZero/MuZero engine using a residential gaming PC, or would it take thousands of years of training for the AI to learn enough to challenge humans?
  
  I'm having trouble wrapping my head around how much computing power '4 hours of Google DeepMind training' equates to my residential computer running 24/7 trying to build a trained AI.
  
  Basically, are AlphaZero or MuZero practical for indie board games that want a state of the art AI, or is it too expensive to train?
  
  **Tags:** training, game-ai, alphazero, muzero
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2021-02-15. Score: 3 | Accepted answer
  
  The vast majority of neural networks are now trained on graphics processing units (GPUs) or specialised accelerator hardware such as [tensor processing units](https://en.wikipedia.org/wiki/Tensor_Processing_Unit) (TPUs).
  
  In [Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm](https://arxiv.org/pdf/1712.01815.pdf), Silver et al. say that the training process involved 5,000 first-generation TPUs generating self-play games and 64 second-generation TPUs for training. This is certainly far beyond what any practical gaming computer is likely to achieve, as you'll likely only have one GPU, and that might not even rival a single TPU. Training on the CPU will be substantially slower again than either a GPU or TPU. Training would be orders of magnitude slower; you might find [these benchmarks](https://arxiv.org/pdf/1907.10701.pdf) by Wang et al. of interest.
  
  > **Comments:**
  > **username_3**: Not arguing with any part of this answer, and I may add that I think gcp (creator of Leela zero) estimated something like 10000 years in single GPU (estimated 3 years ago). However this training time depends on the game complexity a lot and also if the game has no professional players then a single GPU may well be the best in the world after a few months if not a few weeks or even days.  
  
  ---
  
  ### Answer by username_1 on 2022-07-23. Score: 4
  
  For board games, you may not need an advanced computer to achieve very good results. TPU and high-end GPUs allow to go one step further of course.
  
  But there is an existing implementation on github alpha-zero-general (not from me). By adding some optimizations tricks from scientific articles, I managed to train alphazero for several board games (Splendor, Santorini, Machi Koro, The Little Prince Make Me A Planet) with my i5 CPU. You may not need GPU as most of the time is spent in self-playing more than training, meaning that computer needs to do lots of network inferences more than gradient. After about a day of training, the training seemed to reach its asymptote. As a result, the computer was winning over me 100% on each game.
  
  I don't want to promote my self github, just telling OP that is feasible.
  
  
  ---
  
  