Answers.Body: <p>I don't have enough reputation to comment, but <a href="https://medium.com/applied-data-science/alphago-zero-explained-in-one-diagram-365f5abf67e0" rel="nofollow noreferrer">AlphaGo Zero Explained In One Diagram</a> is pretty good.</p>
  
  <p>I also really like <a href="http://web.stanford.edu/~surag/posts/alphazero.html" rel="nofollow noreferrer">this</a> tutorial.</p>
  
  <p>Note that the first link doesn't describe when to create (expand) nodes. That part can be a little confusing. <a href="http://ml4dummies.blogspot.com/2018/01/some-confusion-i-encountered-while.html" rel="nofollow noreferrer">This</a> link may help.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-01-09T14:37:53.303Z
Answers.Text: That's good. I like it. +1.
Answers.UserDisplayName: SmallChess
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-01-09T14:34:33.270Z
Answers.LastActivityDate: 2018-01-10T16:04:00.863Z
Answers.LastEditDate: 2018-01-10T16:04:00.863Z
Answers.UserDisplayName: monk
Answers.Body: <p><a href="https://www.youtube.com/watch?v=Bm7zah_LrmE" rel="nofollow noreferrer">This presentation summarizes the journey to AlphaGo</a></p>
  
  <p><a href="https://www.youtube.com/watch?v=ggqnxyjaKe4&amp;list=PLe0J3_6vYq7vQjZQXLC0lauk9UFOehDXA" rel="nofollow noreferrer">This is a presentation of the underlying theory</a></p>
  
  <p><a href="https://mitpress.mit.edu/books/reinforcement-learning" rel="nofollow noreferrer">This is the book</a></p>
  
  <p>Plus, I guess, anything on deep convolutional neural networks.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-12-10T14:14:11.160Z
Answers.LastActivityDate: 2017-12-10T14:14:11.160Z
Answers.UserDisplayName: IA Petr Harasimovic
Answers.Body: <p><strong>EDIT</strong></p>
  
  <p>@unutbu's link in the comment is a good introductory read.</p>
  
  <hr>
  
  <p>Solid understanding for AlphaZero most likely require a quantitative degree (PhD?). Are you asking for a crash course in AlphaZero?</p>
  
  <p>Please note unless you invest <strong>significant</strong> amount of your time, nothing I say will work. There is no book that can possibly cover everything. You will need to work very hard.</p>
  
  <p>Let's give a try. Crash course for AlphaZero.</p>
  
  <p><strong>1.</strong> Basic understanding in machine learning</p>
  
  <p>Countless number of books. But if you don't have time, I'd recommend Professor Andrew Ng's machine learning courses on Coursera. </p>
  
  <p>Your keywords: <em>training sets, test sets, stochastic gradient descent, learning rate, GPU, cost function, cross entropy</em>.</p>
  
  <p><strong>2.</strong> Deep neutral networks</p>
  
  <p>You need to understand about neural networks. I'd recommend Professor Ian Goodfellow's deep learning book if you're serious. If you don't have time, please  again follow Professor Andrew Ng 's online course on Coursera. You don't have to go though all chapters (but you should!).</p>
  
  <p>YouTube has many quick introduction to neural networks, try them.</p>
  
  <p>Your keywords: <em>neuron, layers, weights, bias, mini-batch, activation</em>.</p>
  
  <p><strong>3.</strong> Monte Carlo Tree Search</p>
  
  <p>You should understand what Monte Carlo is. Books about Monte Carlo is everywhere on Amazon. Read the <a href="https://en.wikipedia.org/wiki/Monte_Carlo_tree_search" rel="nofollow noreferrer">wikipedia</a> about MCTS if you don't have time.</p>
  
  <p>Your keywords: <em>sampling, expansion, simulation, rollouts, backpropagation</em>.</p>
  
  <p><strong>3.</strong> Reinforcement learning</p>
  
  <ul>
  <li><a href="https://www.quora.com/What-is-the-REINFORCE-algorithm" rel="nofollow noreferrer">https://www.quora.com/What-is-the-REINFORCE-algorithm</a></li>
  <li><a href="http://www.scholarpedia.org/article/Policy_gradient_methods" rel="nofollow noreferrer">http://www.scholarpedia.org/article/Policy_gradient_methods</a></li>
  </ul>
  
  <p>Your keywords: <em>policy gradient, gradient descent, learning rate</em></p>
  
  <p><strong>5.</strong> Chess board representation</p>
  
  <p>The paper itself is simplest. The model encodes the board state (e.g. pieces) as a series of one-hot encoding binary values.</p>
  
  <p><strong>6.</strong> MCTS vs alpha-beta</p>
  
  <p><a href="https://i.sstatic.net/UgOts.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/UgOts.png" alt="enter image description here"></a></p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-12-11T10:55:22.010Z
Answers.Text: thanks very much for the detailed answer. So it seems that unfortunately there is no straight forward guide. 
  My point is that of course as a CS major I know about neural networks, but not the specific kind of deep networks that were used in Alpha*. I roughly know about MCTS as introduced by Bernd Brügmann, but not the specific kind of adaptation for Alpha*. Same for reinforcement learning.
  For example Andrew Ng's course on ML is very broad and covers stuff like PCA/LDA, which as far as I know is completely irrelevant here.
  But I guess, I have to work it out myself then ;-)
Answers.UserDisplayName: ndbd
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-12-10T14:09:58.650Z
Answers.LastActivityDate: 2017-12-11T04:58:08.817Z
Answers.LastEditDate: 2017-12-11T04:58:08.817Z
Answers.UserDisplayName: SmallChess
Body: <p>I am very interested in how AlphaGo resp. AlphaZero works. It seems to me, the related Google Papers are very dense and not easy to read.</p>
  
  <p>Is there any textbook or lecture that explains on a technical level how it all works? I.e. starting with neural networks/deep learning, Monte Carlo Tree Search until reaching at AlphaGo/AlphaZero?</p>
  
  <p>I have a master in computer science, but I never worked with artificial intelligence/machine learning.</p>
  
  <p>Right now it looks to me that all the related information are scattered around, and I don't have time to listen to (online) lectures about machine learning that contain maybe only 10% on what is relevant to understand AlphaGo...</p>
  
  <p>Can someone point me in the right direction?</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-12-10T22:21:34.203Z
Comments.Text: [Google DeepMind's AlphaGo: How it works](https://www.tastehit.com/blog/google-deepmind-alphago-how-it-works/)
Comments.UserDisplayName: unutbu
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-12-11T10:49:51.563Z
Comments.Text: thanks for the link, but that presentation is really really superficial
Comments.UserDisplayName: ndbd
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-01-08T16:15:03.953Z
Comments.Text: coming back to this a few years later, I've found https://github.com/asdfjkl/neural_network_chess which comes pretty close to what I was looking for back then.
Comments.UserDisplayName: ndbd
Tags: <chess-algorithms><alphazero>
Title: Lecture/Book on AlphaGo/AlphaZero
ThreadText: # Lecture/Book on AlphaGo/AlphaZero
  
  (Asked by: username_0 on 2017-12-10)
  
  I am very interested in how AlphaGo resp. AlphaZero works. It seems to me, the related Google Papers are very dense and not easy to read.
  
  Is there any textbook or lecture that explains on a technical level how it all works? I.e. starting with neural networks/deep learning, Monte Carlo Tree Search until reaching at AlphaGo/AlphaZero?
  
  I have a master in computer science, but I never worked with artificial intelligence/machine learning.
  
  Right now it looks to me that all the related information are scattered around, and I don't have time to listen to (online) lectures about machine learning that contain maybe only 10% on what is relevant to understand AlphaGo...
  
  Can someone point me in the right direction?
  
  **Tags:** chess-algorithms, alphazero
  
  > **Comments:**
  > **username_1**: [Google DeepMind's AlphaGo: How it works](https://www.tastehit.com/blog/google-deepmind-alphago-how-it-works/)  
  > **username_0**: thanks for the link, but that presentation is really really superficial  
  > **username_0**: coming back to this a few years later, I've found https://github.com/asdfjkl/neural_network_chess which comes pretty close to what I was looking for back then.  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_3 on 2017-12-10. Score: 10 | Accepted answer
  
  **EDIT**
  
  @username_1's link in the comment is a good introductory read.
  
  ---
  
  Solid understanding for AlphaZero most likely require a quantitative degree (PhD?). Are you asking for a crash course in AlphaZero?
  
  Please note unless you invest **significant** amount of your time, nothing I say will work. There is no book that can possibly cover everything. You will need to work very hard.
  
  Let's give a try. Crash course for AlphaZero.
  
  **1.** Basic understanding in machine learning
  
  Countless number of books. But if you don't have time, I'd recommend Professor Andrew Ng's machine learning courses on Coursera.
  
  Your keywords: *training sets, test sets, stochastic gradient descent, learning rate, GPU, cost function, cross entropy*.
  
  **2.** Deep neutral networks
  
  You need to understand about neural networks. I'd recommend Professor Ian Goodfellow's deep learning book if you're serious. If you don't have time, please again follow Professor Andrew Ng 's online course on Coursera. You don't have to go though all chapters (but you should!).
  
  YouTube has many quick introduction to neural networks, try them.
  
  Your keywords: *neuron, layers, weights, bias, mini-batch, activation*.
  
  **3.** Monte Carlo Tree Search
  
  You should understand what Monte Carlo is. Books about Monte Carlo is everywhere on Amazon. Read the [wikipedia](https://en.wikipedia.org/wiki/Monte_Carlo_tree_search) about MCTS if you don't have time.
  
  Your keywords: *sampling, expansion, simulation, rollouts, backpropagation*.
  
  **3.** Reinforcement learning
  
  * <https://www.quora.com/What-is-the-REINFORCE-algorithm>
  * <http://www.scholarpedia.org/article/Policy_gradient_methods>
  
  Your keywords: *policy gradient, gradient descent, learning rate*
  
  **5.** Chess board representation
  
  The paper itself is simplest. The model encodes the board state (e.g. pieces) as a series of one-hot encoding binary values.
  
  **6.** MCTS vs alpha-beta
  
  [![enter image description here](https://i.sstatic.net/UgOts.png)](https://i.sstatic.net/UgOts.png)
  
  > **Comments:**
  > **username_0**: thanks very much for the detailed answer. So it seems that unfortunately there is no straight forward guide.  My point is that of course as a CS major I know about neural networks, but not the specific kind of deep networks that were used in Alpha*. I roughly know about MCTS as introduced by Bernd Brügmann, but not the specific kind of adaptation for Alpha*. Same for reinforcement learning. For example Andrew Ng's course on ML is very broad and covers stuff like PCA/LDA, which as far as I know is completely irrelevant here. But I guess, I have to work it out myself then ;-)  
  
  ---
  
  ### Answer by username_2 on 2018-01-09. Score: 7
  
  I don't have enough reputation to comment, but [AlphaGo Zero Explained In One Diagram](https://medium.com/applied-data-science/alphago-zero-explained-in-one-diagram-365f5abf67e0) is pretty good.
  
  I also really like [this](http://web.stanford.edu/~surag/posts/alphazero.html) tutorial.
  
  Note that the first link doesn't describe when to create (expand) nodes. That part can be a little confusing. [This](http://ml4dummies.blogspot.com/2018/01/some-confusion-i-encountered-while.html) link may help.
  
  > **Comments:**
  > **username_3**: That's good. I like it. +1.  
  
  ---
  
  ### Answer by username_4 on 2017-12-10. Score: 3
  
  [This presentation summarizes the journey to AlphaGo](https://www.youtube.com/watch?v=Bm7zah_LrmE)
  
  [This is a presentation of the underlying theory](https://www.youtube.com/watch?v=ggqnxyjaKe4&list=PLe0J3_6vYq7vQjZQXLC0lauk9UFOehDXA)
  
  [This is the book](https://mitpress.mit.edu/books/reinforcement-learning)
  
  Plus, I guess, anything on deep convolutional neural networks.
  
  
  ---
  
  