Answers.Body: <p>Your observation is certainly correct: Assuming the chess boards gets flipped to the point of view of the current player, and depending on how exactly you handle promotions, castling, and other special moves there are about 2000 distinct chess moves. This means that about half of the policy output is never useful.</p>
  <p>However, as stated in the paper:</p>
  <blockquote>
  <p>Illegal moves are masked out by setting their probabilities to zero, and re-normalising the probabilities for remaining moves.</p>
  </blockquote>
  <p>This means those useless policy output values don't hurt, since they always get set to zero. The network doesn't even have to learn that those are illegal moves, it gets that for free.</p>
  <p>There is the small effect of having to compute some useless values, but the output head is only a tiny fraction of the total compute required by the network, so it can safely be ignored. If you're really worried about this you can use a fully connected output layer for the policy that only encodes potentially valid moves, this is what some earlier AlphaZero versions did.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-02T22:36:20.523Z
Answers.LastActivityDate: 2023-01-02T22:36:20.523Z
Answers.UserDisplayName: KarelPeeters
Body: <p>The AlphaZero research team states</p>
  <blockquote>
  <p>A move in chess may be described in two parts: selecting the piece to move, and then
  selecting among the legal moves for that piece. We represent the policy π(a|s) by a 8 × 8 × 73
  stack of planes encoding a probability distribution over 4,672 possible moves. Each of the 8×8
  positions identifies the square from which to “pick up” a piece.</p>
  </blockquote>
  <p>I am wondering if it would be better to only have the network output moves that could exist (i.e., &quot;a1b3&quot; is a possible knight move, but &quot;a1g3&quot; could never be reached by a piece).
  The modified output would be much smaller, and could potentially make the neural network learn quicker, right?</p>
  
Tags: <alphazero>
Title: Is AlphaZero's output (action probabilities) vector suboptimal?
UserDisplayName: ace.exe
ThreadText: # Is AlphaZero's output (action probabilities) vector suboptimal?
  
  (Asked by: username_0 on 2023-01-02)
  
  The AlphaZero research team states
  
  > A move in chess may be described in two parts: selecting the piece to move, and then
  > selecting among the legal moves for that piece. We represent the policy π(a|s) by a 8 × 8 × 73
  > stack of planes encoding a probability distribution over 4,672 possible moves. Each of the 8×8
  > positions identifies the square from which to “pick up” a piece.
  
  I am wondering if it would be better to only have the network output moves that could exist (i.e., "a1b3" is a possible knight move, but "a1g3" could never be reached by a piece).
  The modified output would be much smaller, and could potentially make the neural network learn quicker, right?
  
  **Tags:** alphazero
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-01-02. Score: 1 | Accepted answer
  
  Your observation is certainly correct: Assuming the chess boards gets flipped to the point of view of the current player, and depending on how exactly you handle promotions, castling, and other special moves there are about 2000 distinct chess moves. This means that about half of the policy output is never useful.
  
  However, as stated in the paper:
  
  > Illegal moves are masked out by setting their probabilities to zero, and re-normalising the probabilities for remaining moves.
  
  This means those useless policy output values don't hurt, since they always get set to zero. The network doesn't even have to learn that those are illegal moves, it gets that for free.
  
  There is the small effect of having to compute some useless values, but the output head is only a tiny fraction of the total compute required by the network, so it can safely be ignored. If you're really worried about this you can use a fully connected output layer for the policy that only encodes potentially valid moves, this is what some earlier AlphaZero versions did.
  
  
  ---
  
  