Answers.Body: <p>I don't know how to play Go at all, but the paper wrote:</p>
  <blockquote>
  <p>History features Xt, Yt are necessary, because Go is not fully
  observable solely from the current stones, as repetitions are
  forbidden</p>
  </blockquote>
  <p>For chess, I'm unable find any mention about it in the paper other than:</p>
  <blockquote>
  <p>Unless otherwise specified, the training and search algorithm and
  parameters are identical to AlphaGo Zero (29).</p>
  </blockquote>
  <p>The training process was identical, but the sentence made no mention about the input features.</p>
  <p>If you study the papers more carefully, the input features for Go is:</p>
  <pre><code>st = [Xt, Yt, Xt−1, Yt−1,..., Xt−7, Yt−7, C]
  </code></pre>
  <p>It's a representation only for Go, not chess. The input features for chess outlined in the paper covers 50-move rules and repetitions already, so there isn't any need for time steps in the input layer.</p>
  <p>Google didn't use timesteps for chess, but only for Go.</p>
  <blockquote>
  <p>Also, the architecture that I'm trying to build is not a Reinforcement
  Learning model, but rather a supervised learning one based on the FICS
  game database of players with high ELO. Basically, I will train 2
  separate models, one for white and one for black. Data for the white
  NN is taken from games where white wins, black NN takes black wins,
  drawn games are used for both, so that white moves in white winning
  games are considered &quot;good&quot;, and vice versa. Is this a reasonable
  approach?</p>
  </blockquote>
  <p>Supervised learning is ok. It doesn't make sense to me for an ensemble of two models here. Why not just model it as a probability from 0 to 1?</p>
  <p><strong>EDIT:</strong></p>
  <p>You actually do need the entire game history in chess, for 50-moves and repetitions.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-16T01:24:15.670Z
Answers.Text: Thanks for the answer. I thought so too as a chess player doesn't need move history to make a decision. As for the number of models, I use 2 instead of 1 because white's moves in white winning games are labelled as 'good' whereas in the same game black moves can be either good or bad (maybe they were good but white was better to actually win)  hence impossible to provide a label
Answers.UserDisplayName: dqtvictory
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-16T01:47:11.230Z
Answers.Text: Well, you do but only for repetitions and 50-moves.
Answers.UserDisplayName: SmallChess
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-16T10:37:17.120Z
Answers.Text: For 50 moves, a single number should be enough information (number of ply since last pawn move / capture). For detection of threefold repetition some history is needed.
Answers.UserDisplayName: RemcoGerlich
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-16T20:02:10.583Z
Answers.Text: In Go, pieces are placed on the board (starting with an empty board), and may be captured and removed from the board, but they are never moved in any other way. All variants of Go include the so-called "ko rule" which prevents the *immediate* recapture (on the next move) of an opponent's piece, producing the same board position as before the opponent's move. Some variants have a more general "superko rule" that a move which creates a position identical to *any* earlier board position in the game is illegal. For either rule, a representation which stores the *order* of all moves is sufficient.
Answers.UserDisplayName: alephzero
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-02-27T21:55:53.780Z
Answers.Text: Umm, they actually did use history in the AlphaZero chess. Table S1 of  their paper "Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm" shows a total of 119 input planes, most of which come from representation of the board history.  This is despite them having a no-progress-moves count input plane.  It's really unclear to me why they matched the Go architecture and included history.
Answers.UserDisplayName: Ben G
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-15T19:03:38.573Z
Answers.LastActivityDate: 2020-11-16T02:03:34.860Z
Answers.LastEditDate: 2020-11-16T02:03:34.860Z
Answers.UserDisplayName: SmallChess
Body: <p>I'm trying to build my own chess engine using neural nets, and have been researching papers from Deepmind about their algorithm implemented in AlphaZero. Since chess is a game with perfect information (meaning the only unknown information in the game is what's going on in the opponent's head) and that each action resulting from a game state does not depend on the previous actions (except castling rules for example), why does AlphaZero have an 8-step temporal implementation in its input layer? In other words, its NN, aside from looking at the current game state, also looks at states from 8 previous moves in order to make a decision? AlphaZero's paper didn't have a say about this.</p>
  <p>Also, the architecture that I'm trying to build is not a Reinforcement Learning model, but rather a supervised learning one based on the FICS game database of players with high ELO. Basically, I will train 2 separate models, one for white and one for black. Data for the white NN is taken from games where white wins, black NN takes black wins, drawn games are used for both, so that white moves in white winning games are considered &quot;good&quot;, and vice versa. Is this a reasonable approach?</p>
  
Tags: <engines><software><programming><alphazero>
Title: Why does AlphaZero implement time steps in its input layers?
ThreadText: # Why does AlphaZero implement time steps in its input layers?
  
  (Asked by: username_0 on 2020-11-15)
  
  I'm trying to build my own chess engine using neural nets, and have been researching papers from Deepmind about their algorithm implemented in AlphaZero. Since chess is a game with perfect information (meaning the only unknown information in the game is what's going on in the opponent's head) and that each action resulting from a game state does not depend on the previous actions (except castling rules for example), why does AlphaZero have an 8-step temporal implementation in its input layer? In other words, its NN, aside from looking at the current game state, also looks at states from 8 previous moves in order to make a decision? AlphaZero's paper didn't have a say about this.
  
  Also, the architecture that I'm trying to build is not a Reinforcement Learning model, but rather a supervised learning one based on the FICS game database of players with high ELO. Basically, I will train 2 separate models, one for white and one for black. Data for the white NN is taken from games where white wins, black NN takes black wins, drawn games are used for both, so that white moves in white winning games are considered "good", and vice versa. Is this a reasonable approach?
  
  **Tags:** engines, software, programming, alphazero
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-11-15. Score: 1
  
  I don't know how to play Go at all, but the paper wrote:
  
  History features Xt, Yt are necessary, because Go is not fully
  observable solely from the current stones, as repetitions are
  forbidden
  
  For chess, I'm unable find any mention about it in the paper other than:
  
  Unless otherwise specified, the training and search algorithm and
  parameters are identical to AlphaGo Zero (29).
  
  The training process was identical, but the sentence made no mention about the input features.
  If you study the papers more carefully, the input features for Go is:
  st = [Xt, Yt, Xt−1, Yt−1,..., Xt−7, Yt−7, C]
  
  It's a representation only for Go, not chess. The input features for chess outlined in the paper covers 50-move rules and repetitions already, so there isn't any need for time steps in the input layer.
  Google didn't use timesteps for chess, but only for Go.
  
  Also, the architecture that I'm trying to build is not a Reinforcement
  Learning model, but rather a supervised learning one based on the FICS
  game database of players with high ELO. Basically, I will train 2
  separate models, one for white and one for black. Data for the white
  NN is taken from games where white wins, black NN takes black wins,
  drawn games are used for both, so that white moves in white winning
  games are considered "good", and vice versa. Is this a reasonable
  approach?
  
  Supervised learning is ok. It doesn't make sense to me for an ensemble of two models here. Why not just model it as a probability from 0 to 1?
  EDIT:
  You actually do need the entire game history in chess, for 50-moves and repetitions.
  
  > **Comments:**
  > **username_0**: Thanks for the answer. I thought so too as a chess player doesn't need move history to make a decision. As for the number of models, I use 2 instead of 1 because white's moves in white winning games are labelled as 'good' whereas in the same game black moves can be either good or bad (maybe they were good but white was better to actually win)  hence impossible to provide a label  
  > **username_1**: Well, you do but only for repetitions and 50-moves.  
  > **username_2**: For 50 moves, a single number should be enough information (number of ply since last pawn move / capture). For detection of threefold repetition some history is needed.  
  > **username_3**: In Go, pieces are placed on the board (starting with an empty board), and may be captured and removed from the board, but they are never moved in any other way. All variants of Go include the so-called "ko rule" which prevents the *immediate* recapture (on the next move) of an opponent's piece, producing the same board position as before the opponent's move. Some variants have a more general "superko rule" that a move which creates a position identical to *any* earlier board position in the game is illegal. For either rule, a representation which stores the *order* of all moves is sufficient.  
  > **username_4**: Umm, they actually did use history in the AlphaZero chess. Table S1 of  their paper "Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm" shows a total of 119 input planes, most of which come from representation of the board history.  This is despite them having a no-progress-moves count input plane.  It's really unclear to me why they matched the Go architecture and included history.  
  
  ---
  
  