Answers.Body: <h1>The primary questions</h1>
  <ol>
  <li><p>They only trained AlphaZero once, and then let it play against <em>itself</em>. So yes, both players are identical.</p>
  </li>
  <li><p>MCTS as implemented by AlphaZero does not intentionally use any randomness <em>during tournament play</em>. There is probably a bit of inherent randomness from multithreading, but no <em>seed</em> as such. I talk about this is a bit more in <a href="https://ai.stackexchange.com/a/34266/52372">this answer</a> and its comments.</p>
  </li>
  </ol>
  <h1>Different A0 vs A0 games</h1>
  <p>Something you seem to be missing is that typically when engine vs engine matches are played an <a href="https://www.chessprogramming.org/Opening_Book" rel="nofollow noreferrer">opening book</a> is used. This is mostly to ensure game diversity, since many engines are (almost) deterministic, so you'd just end up with a single game. The video you linked links to a <a href="https://www.chess.com/article/view/no-castling-chess-kramnik-alphazero" rel="nofollow noreferrer">chess.com article</a>, with some more details:</p>
  <ul>
  <li>
  <blockquote>
  <p>Computer Match, selfplay from move 4 pg</p>
  </blockquote>
  </li>
  <li>
  <blockquote>
  <p><code>1. e4 c62 2. d4 d5</code> the last book move</p>
  </blockquote>
  </li>
  </ul>
  <p>So the first 4 moves of the game where set by humans, only afterwards AlphaZero started to play by itself. This means you can get as many different games as you want, in fact at the bottom of the article there are a couple more with different opening moves.</p>
  <h1>Different playing styles for different seeds</h1>
  <p>This is hard to answer conclusively, since &quot;playing styles&quot; are a bit of a nebulous concept. I've trained smaller scale versions of A0 myself, and after training for a while they always end up around the same playing strength for the same hyperparameters.</p>
  <p>Intuitively I think that different network initialization seeds would not really cause different playing styles, or at least that any differences disappear after a training for a while.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-02-27T11:49:30.063Z
Answers.LastActivityDate: 2022-03-10T21:30:40.203Z
Answers.LastEditDate: 2022-03-10T21:30:40.203Z
Answers.UserDisplayName: KarelPeeters
Body: <p>There is a quite popular <a href="https://www.youtube.com/watch?v=mOqmLYlFdBo" rel="nofollow noreferrer">video</a> analysing a chess game <strong>AlphaZero vs. AlphaZero</strong>, called &quot;the perfect game&quot;. It leaves some questions open and I'd like to ask them here:</p>
  <ol>
  <li><p>Did the two copies of AlphaZero use the same random seeds in the <a href="https://arxiv.org/abs/2111.09259" rel="nofollow noreferrer">learning phase</a>, esp. when generating random games? So were they perfect copies of each other at the beginning of the game?</p>
  </li>
  <li><p>Do the two copies of AlphaZero calculate their moves with the same random seeds when doing <a href="https://joshvarty.github.io/AlphaZero/" rel="nofollow noreferrer">Monte Carlo tree search</a>?</p>
  </li>
  </ol>
  <p>In any case there are three possible games all of which may be called &quot;AlphaZero vs. AlphaZero&quot;:</p>
  <ol>
  <li>same seeds when learning, same seeds when playing</li>
  <li>same seeds when learning, different seeds when playing</li>
  <li>different seeds when learning (the seeds when playing don't matter then)</li>
  </ol>
  <p>In the latter case, I wonder if AlphaZero's playing style (which is for example analyzed in the book <a href="https://www.newinchess.com/game-changer" rel="nofollow noreferrer">Game Changer</a> by Matthew Sadler and Natasha Regan) may depend on the seeds used for generating random games (assuming that the same number of test games is played in the learning phase). In other words: <strong>Can AlphaZero develop significantly different playing styles</strong> when starting tabula rasa? In this case, Sadler/Regan's book would describe just one instance of AlphaZero.</p>
  <p>The last question should be answerable also by those of you who don't know the video and how the premises have been.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-02-24T13:53:23.287Z
Comments.Text: Can you please put your **specific** question in the title? "AlphaZero vs. AlphaZero" is not a question and is it's not specific.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-02-24T15:57:59.217Z
Comments.Text: @nbro: Done. Sorry again, I will learn it;-)
Comments.UserDisplayName: Hans-Peter Stricker
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-02-26T07:39:26.000Z
Comments.Text: I'm fairly certain that it's just the same copy of alpha zero, playing against itself.
Comments.UserDisplayName: Taw
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-02-26T07:41:35.833Z
Comments.Text: The strength of alpha zero is proportional to how much it has been trained; so if you had two copies of it that were trained for a different number of episodes, or two copies which used different hyperparameters, they may end up at different ELO strengths. But it's awfully hard to imagine that they would have different playing styles, except perhaps at very low, sub human elo due to randomness from initialization
Comments.UserDisplayName: Taw
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-02-26T10:16:36.567Z
Comments.Text: @Taw: Thanks a lot, this helps. Can you tell me where I can find a concise list of AlphaZero's *hyper*parameters? In the [original paper](https://arxiv.org/pdf/1712.01815.pdf) only parameters $\theta$ are mentioned (leaving open which exactly these are), and there is no talk of *hyper*parameters. Could you please clarify?
Comments.UserDisplayName: Hans-Peter Stricker
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-02-26T23:50:05.727Z
Comments.Text: @Hans I don't think the alphazero code was ever released, so no one really knows exactly, except for the authors. If you want to study this more, I would look into one of the public codebases (which are not released by deepmind, but are based off their work). These include leela chess zero https://github.com/LeelaChessZero/lc0
  which is the strongest alphazero based chess engine currently. Leela is stronger than the original alpha zero.
Comments.UserDisplayName: Taw
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-02-26T23:52:15.877Z
Comments.Text: More generally I believe there are many important hyperparamters such as the depth, width, of the MCTS, how the branching is done, and how to update the value estimate based on MCTS. Then of course there would be the 'standard' hyperparamters such as learning rate, batch size, size of memory replay etc. which probably don't make as much of a difference in terms of the playing style.
Comments.UserDisplayName: Taw
Tags: <monte-carlo-tree-search><alphazero><chess><explainable-ai><randomness>
Title: Can AlphaZero develop significantly different playing styles (depending on the random games from which it learrns)?
UserDisplayName: Hans-Peter Stricker
ThreadText: # Can AlphaZero develop significantly different playing styles (depending on the random games from which it learrns)?
  
  (Asked by: username_0 on 2022-02-24)
  
  There is a quite popular [video](https://www.youtube.com/watch?v=mOqmLYlFdBo) analysing a chess game **AlphaZero vs. AlphaZero**, called "the perfect game". It leaves some questions open and I'd like to ask them here:
  
  1. Did the two copies of AlphaZero use the same random seeds in the [learning phase](https://arxiv.org/abs/2111.09259), esp. when generating random games? So were they perfect copies of each other at the beginning of the game?
  2. Do the two copies of AlphaZero calculate their moves with the same random seeds when doing [Monte Carlo tree search](https://joshvarty.github.io/AlphaZero/)?
  
  In any case there are three possible games all of which may be called "AlphaZero vs. AlphaZero":
  
  1. same seeds when learning, same seeds when playing
  2. same seeds when learning, different seeds when playing
  3. different seeds when learning (the seeds when playing don't matter then)
  
  In the latter case, I wonder if AlphaZero's playing style (which is for example analyzed in the book [Game Changer](https://www.newinchess.com/game-changer) by Matthew Sadler and Natasha Regan) may depend on the seeds used for generating random games (assuming that the same number of test games is played in the learning phase). In other words: **Can AlphaZero develop significantly different playing styles** when starting tabula rasa? In this case, Sadler/Regan's book would describe just one instance of AlphaZero.
  
  The last question should be answerable also by those of you who don't know the video and how the premises have been.
  
  **Tags:** monte-carlo-tree-search, alphazero, chess, explainable-ai, randomness
  > **Comments:**
  > **username_1**: Can you please put your **specific** question in the title? "AlphaZero vs. AlphaZero" is not a question and is it's not specific.  
  > **username_0**: @nbro: Done. Sorry again, I will learn it;-)  
  > **username_2**: I'm fairly certain that it's just the same copy of alpha zero, playing against itself.  
  > **username_2**: The strength of alpha zero is proportional to how much it has been trained; so if you had two copies of it that were trained for a different number of episodes, or two copies which used different hyperparameters, they may end up at different ELO strengths. But it's awfully hard to imagine that they would have different playing styles, except perhaps at very low, sub human elo due to randomness from initialization  
  > **username_0**: @Taw: Thanks a lot, this helps. Can you tell me where I can find a concise list of AlphaZero's *hyper*parameters? In the [original paper](https://arxiv.org/pdf/1712.01815.pdf) only parameters $\theta$ are mentioned (leaving open which exactly these are), and there is no talk of *hyper*parameters. Could you please clarify?  
  > **username_2**: @Hans I don't think the alphazero code was ever released, so no one really knows exactly, except for the authors. If you want to study this more, I would look into one of the public codebases (which are not released by deepmind, but are based off their work). These include leela chess zero https://github.com/LeelaChessZero/lc0 which is the strongest alphazero based chess engine currently. Leela is stronger than the original alpha zero.  
  > **username_2**: More generally I believe there are many important hyperparamters such as the depth, width, of the MCTS, how the branching is done, and how to update the value estimate based on MCTS. Then of course there would be the 'standard' hyperparamters such as learning rate, batch size, size of memory replay etc. which probably don't make as much of a difference in terms of the playing style.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2022-02-27. Score: 1
  
  # The primary questions
  
  1. They only trained AlphaZero once, and then let it play against *itself*. So yes, both players are identical.
  2. MCTS as implemented by AlphaZero does not intentionally use any randomness *during tournament play*. There is probably a bit of inherent randomness from multithreading, but no *seed* as such. I talk about this is a bit more in [this answer](https://ai.stackexchange.com/a/34266/52372) and its comments.
  
  # Different A0 vs A0 games
  
  Something you seem to be missing is that typically when engine vs engine matches are played an [opening book](https://www.chessprogramming.org/Opening_Book) is used. This is mostly to ensure game diversity, since many engines are (almost) deterministic, so you'd just end up with a single game. The video you linked links to a [chess.com article](https://www.chess.com/article/view/no-castling-chess-kramnik-alphazero), with some more details:
  
  * > Computer Match, selfplay from move 4 pg
  * > `1. e4 c62 2. d4 d5` the last book move
  
  So the first 4 moves of the game where set by humans, only afterwards AlphaZero started to play by itself. This means you can get as many different games as you want, in fact at the bottom of the article there are a couple more with different opening moves.
  
  # Different playing styles for different seeds
  
  This is hard to answer conclusively, since "playing styles" are a bit of a nebulous concept. I've trained smaller scale versions of A0 myself, and after training for a while they always end up around the same playing strength for the same hyperparameters.
  
  Intuitively I think that different network initialization seeds would not really cause different playing styles, or at least that any differences disappear after a training for a while.
  
  
  ---
  
  