Answers.Body: <p>You should have a method to generate a possible moves output based on the board state. Use this as a mask before normalization in the policy head.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-10T12:27:27.000Z
Answers.LastActivityDate: 2019-07-10T20:46:24.560Z
Answers.LastEditDate: 2019-07-10T20:46:24.560Z
Answers.UserDisplayName: mshlis
Body: <p>I am interested in creating a neural network-based engine for chess. It uses a <span class="math-container">$8 \times 8 \times 73$</span> output space for each possible move as proposed in the Alpha Zero paper: <a href="https://arxiv.org/pdf/1712.01815.pdf" rel="nofollow noreferrer">Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm</a>.</p>
  
  <p>However, when running the network, the first selected move is invalid. How should we deal with this? Basically, I see two options.</p>
  
  <ol>
  <li>Pick the next highest outputted move, until it is a valid move. In this case, the network might automatically over time not put illegal moves on top.</li>
  <li>Process the game as a loss for the player who picked the illegal move. This might have the disadvantage that the network might be 'stuck' on only a few legal moves.</li>
  </ol>
  
  <p>What is the preferred solution to this particular problem?</p>
  
Tags: <neural-networks><ai-design><game-ai><chess><alphazero>
Title: How to deal with invalid output in a policy network?
UserDisplayName: whits
ThreadText: # How to deal with invalid output in a policy network?
  
  (Asked by: username_0 on 2019-06-10)
  
  I am interested in creating a neural network-based engine for chess. It uses a $8 \times 8 \times 73$ output space for each possible move as proposed in the Alpha Zero paper: [Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm](https://arxiv.org/pdf/1712.01815.pdf).
  
  However, when running the network, the first selected move is invalid. How should we deal with this? Basically, I see two options.
  
  1. Pick the next highest outputted move, until it is a valid move. In this case, the network might automatically over time not put illegal moves on top.
  2. Process the game as a loss for the player who picked the illegal move. This might have the disadvantage that the network might be 'stuck' on only a few legal moves.
  
  What is the preferred solution to this particular problem?
  
  **Tags:** neural-networks, ai-design, game-ai, chess, alphazero
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-06-10. Score: 0
  
  You should have a method to generate a possible moves output based on the board state. Use this as a mask before normalization in the policy head.
  
  
  ---
  
  