Answers.Body: <p>The technique used by AlphaGo is "Monte Carlo Tree Search", combined with a very well trained neural network. The network's job is to estimate the quality of different board states and moves. This estimation is deterministic. If you show AlphaGo the same board on two different occasions, it thinks it is exactly as good (or bad) on both occasions.</p>
  
  <p><a href="https://en.wikipedia.org/wiki/Monte_Carlo_tree_search" rel="noreferrer">Monte Carlo Tree Search</a> however, is a <em>randomized</em> algorithm. So as a simplified explanation, the way AlphaGo decides which moves to make is:</p>
  
  <ol>
  <li>Look at the current board.</li>
  <li>Pick a random move, and imagine what the board would look like if you made that move.</li>
  <li>Pick a random move for your opponent, and imagine what the board would look like if they made that move.</li>
  <li>Keep doing steps 2 &amp; 3 for a while, so that we're imagining being at some board many moves in the future along a random line of play.</li>
  <li>Ask the neural network how good this board state is.</li>
  <li>Repeat steps 1-5 many times. Then make whichever move led to the best lines of play <em>on average</em>, to make right now.</li>
  </ol>
  
  <p>What this means is, AlphaGo won't always play the same way, because it doesn't actually consider every move explicitly. It just thinks about enough lines of play to be pretty confident about whether one move is better than another. This is actually not so far removed from how humans play most of the middle of games like these.</p>
  
  <p>So, in game 4, essentially, <strong>Sedol got lucky</strong>. The random lines of play that AlphaGo chose to look at did not capture some critical facts about one or more board states. This led it to make a mistake. If you asked it to play the same game through again, it might not make the same mistakes (it might think about different random lines of play, that <em>do</em> capture the critical facts it missed in the first game). Further, it might choose to play slightly differently on other moves, which could have a big impact on the rest of the game. These two factors prevented Sedol from simply playing game 4 over again.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-27T04:01:42.990Z
Answers.LastActivityDate: 2019-03-27T04:01:42.990Z
Answers.UserDisplayName: John Doucette
Body: <p>In the documentary about the match, it is said that after losing the 4th game, AlphaGo came back stronger and started to play in a weird way (not human-like) and it was pretty impossible to be beaten. Why and how did that happen? </p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-03-26T20:56:08.433Z
Comments.Text: This might actually be better as a "soft question" since I'm not sure we can know definitively.  The simple answer is that Go is still unsolvable, so both Sedol and AlphaGo were surely making plenty of "mistakes" (non-optimal placements) and, Sedol's choices were stronger in game 4, MCTS is relentless in chipping away at any advantage.  *(I discovered this recently playing Taverner's AIAI in partisan sudoku--for the strong AI, errors aren't an insurmountable obstacle to play out from under.)*
Comments.UserDisplayName: DukeZhou
Tags: <reinforcement-learning><game-ai><monte-carlo-tree-search><alphago>
Title: Why didn't champion of the Go game manage to win the last game against AlphaGo, after winning the 4th one?
UserDisplayName: Jay Critch
ThreadText: # Why didn't champion of the Go game manage to win the last game against AlphaGo, after winning the 4th one?
  
  (Asked by: username_0 on 2019-03-26)
  
  In the documentary about the match, it is said that after losing the 4th game, AlphaGo came back stronger and started to play in a weird way (not human-like) and it was pretty impossible to be beaten. Why and how did that happen?
  
  **Tags:** reinforcement-learning, game-ai, monte-carlo-tree-search, alphago
  > **Comments:**
  > **username_1**: This might actually be better as a "soft question" since I'm not sure we can know definitively.  The simple answer is that Go is still unsolvable, so both Sedol and AlphaGo were surely making plenty of "mistakes" (non-optimal placements) and, Sedol's choices were stronger in game 4, MCTS is relentless in chipping away at any advantage.  *(I discovered this recently playing Taverner's AIAI in partisan sudoku--for the strong AI, errors aren't an insurmountable obstacle to play out from under.)*  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2019-03-27. Score: 6
  
  The technique used by AlphaGo is "Monte Carlo Tree Search", combined with a very well trained neural network. The network's job is to estimate the quality of different board states and moves. This estimation is deterministic. If you show AlphaGo the same board on two different occasions, it thinks it is exactly as good (or bad) on both occasions.
  
  [Monte Carlo Tree Search](https://en.wikipedia.org/wiki/Monte_Carlo_tree_search) however, is a *randomized* algorithm. So as a simplified explanation, the way AlphaGo decides which moves to make is:
  
  1. Look at the current board.
  2. Pick a random move, and imagine what the board would look like if you made that move.
  3. Pick a random move for your opponent, and imagine what the board would look like if they made that move.
  4. Keep doing steps 2 & 3 for a while, so that we're imagining being at some board many moves in the future along a random line of play.
  5. Ask the neural network how good this board state is.
  6. Repeat steps 1-5 many times. Then make whichever move led to the best lines of play *on average*, to make right now.
  
  What this means is, AlphaGo won't always play the same way, because it doesn't actually consider every move explicitly. It just thinks about enough lines of play to be pretty confident about whether one move is better than another. This is actually not so far removed from how humans play most of the middle of games like these.
  
  So, in game 4, essentially, **Sedol got lucky**. The random lines of play that AlphaGo chose to look at did not capture some critical facts about one or more board states. This led it to make a mistake. If you asked it to play the same game through again, it might not make the same mistakes (it might think about different random lines of play, that *do* capture the critical facts it missed in the first game). Further, it might choose to play slightly differently on other moves, which could have a big impact on the rest of the game. These two factors prevented Sedol from simply playing game 4 over again.
  
  
  ---
  
  