Answers.Body: <p>It could be because there is simply not enough data for the late game. To make the model give more importance to the later stages of the game you can try to tweak the loss function such that it penalizes more for when there are fewer pieces on the board. ( This might give an idea on that: <a href="https://medium.com/visionwizard/understanding-focal-loss-a-quick-read-b914422913e7" rel="nofollow noreferrer">https://medium.com/visionwizard/understanding-focal-loss-a-quick-read-b914422913e7</a>)</p>
  <p>One other thing you can do; given that my above-mentioned reasoning is correct, is to provide the number of pieces on the board as an input. I am not sure how you might go around doing that in the one-hot encoded scenario but might be worth looking into (You could append the number of pieces in the binary format to the flattened input). One argument against this is that the network should implicitly know how many pieces are on the board but yet it could still be useful to give it as an extra input.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-01-18T16:07:43.797Z
Answers.LastActivityDate: 2022-01-18T16:07:43.797Z
Answers.UserDisplayName: Nazmican
Answers.Body: <p>I tried hacking together several NN approaches for chess, in the context of <a href="https://github.com/SebLague/Chess-Challenge" rel="nofollow noreferrer">Chess Coding Challenge</a> (implement a chess bot in a relatively small amount of code). I also play online chess semi-regularly, so I have some understanding of the game. I found that in general the more accurate my network was with evaluation (and the more complex it was), the worse it played since it ended up in uncommon situations which it grossly mis-evaluated to its favor.</p>
  <p>I haven't done a detailed analysis, but I suspect that there two main factors here:</p>
  <ul>
  <li>Lack of diversity in the training data, for example does your network correctly evaluate a starting position with queen odds?</li>
  <li>The difficulty of evaluating positions which involve tactics</li>
  </ul>
  <p>Here are three similar positions, sorry it is always <strong>black's turn to move</strong>, since I encountered these when playing as black:</p>
  <p><a href="https://i.sstatic.net/y1VNa.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/y1VNa.png" alt="chess-board-1" /></a></p>
  <p>Evaluation: -6.4</p>
  <p><a href="https://i.sstatic.net/4oxJr.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/4oxJr.png" alt="chess-board-2" /></a></p>
  <p>Evaluation: -3.3</p>
  <p><a href="https://i.sstatic.net/GE90Z.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/GE90Z.png" alt="chess-board-3" /></a></p>
  <p>Evaluation: -2.2</p>
  <p>I don't know a good solution on how to correctly evaluate tactics, except searching explicit steps in the game tree. Note the first position has a rather simple two-move tactic, but modern engines will do searches tens of moves deep.</p>
  <p>And regarding the first issue, I got better results by imposing strong regularization to the network's architecture and weights. For example a good starting point would be an evaluation function which resembles a <a href="https://www.chessprogramming.org/Piece-Square_Tables" rel="nofollow noreferrer">Piece-Square-Table</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-10-04T15:47:40.033Z
Answers.LastActivityDate: 2023-10-04T20:07:27.843Z
Answers.LastEditDate: 2023-10-04T20:07:27.843Z
Answers.UserDisplayName: NikoNyrh
Body: <p>I've been trying to use a Stockfish-like chess evaluation neural network for the past few weeks but to no avail. I wanted to get some other opinions about why my current methods haven't worked.</p>
  <p>Input: <span class="math-container">$8 \times 8 \times 12$</span> one-hot encoded board of pieces. I have around ~60 million unique examples. Evaluations are placed between -15 and 15 (anything greater or smaller is brought to 15 or -15)</p>
  <p>Output: Single number representing Stockfish evaluation.</p>
  <p>I've tried both fully connected and convolutional models and neither have worked very well. Here's some Tensorflow code that might give an idea of what the structures looked like:</p>
  <p>Convolutional model:</p>
  <pre><code>import tensorflow as tf
  
  def block(filters, x, res=None):
    if res is not None:
      x = tf.keras.layers.concatenate([x, res])
    x = tf.keras.layers.Conv2D(filters=filters, kernel_size=3, padding='same', activation='relu', kernel_initializer='he_normal')(x)
    x = tf.keras.layers.BatchNormalization()(x)
    x = tf.keras.layers.MaxPooling2D(pool_size=2, strides=1, padding='same')(x)
    
    return x
  
  board = tf.keras.Input(shape=(8, 8, 12))
  conv1 = block(64, board)
  conv2 = block(128, conv1)
  conv3 = block(256, conv2)
  conv4 = block(512, conv3, conv3)
  conv5 = block(1024, conv4, conv2)
  conv6 = block(1024, conv5, conv1)
  conv7 = block(512, conv6)
  x = tf.keras.layers.Flatten()(conv7)
  x = tf.keras.layers.Dense(128, activation='relu')(x)
  x = tf.keras.layers.Dense(32, activation='relu')(x)
  x = tf.keras.layers.Dense(1)(x)
  </code></pre>
  <p>Fully connected model:</p>
  <pre><code>import tensorflow as tf
  
  board = tf.keras.Input(shape=(8, 8, 12))
  x = tf.keras.layers.Flatten()(board)
  x = tf.keras.layers.Dense(1024, activation='relu')(x)
  x = tf.keras.layers.Dense(1024, activation='relu')(x)
  x = tf.keras.layers.Dense(1024, activation='relu', kernel_regularizer='l2')(x)
  x = tf.keras.layers.Dropout(0.3)(x)
  x = tf.keras.layers.Dense(1024, activation='relu', kernel_regularizer='l2')(x)
  x = tf.keras.layers.Dropout(0.3)(x)
  x = tf.keras.layers.Dense(1024, activation='relu', kernel_regularizer='l2')(x)
  x = tf.keras.layers.Dropout(0.3)(x)
  x = tf.keras.layers.Dense(1024, activation='relu')(x)
  x = tf.keras.layers.Dense(1024, activation='relu')(x)
  x = tf.keras.layers.Dense(1024, activation='relu')(x)
  x = tf.keras.layers.Dense(256, activation='relu')(x)
  x = tf.keras.layers.Dense(1)(x)
  </code></pre>
  <p>Just to add a few notes about what the current results are, they are decent (e.g. mean squared error of Stockfish evaluated cut off between -15 and 15 is about 7). The early game is also played fairly decently but the late game gets really bad (the engine can't evaluate that bishop takes enemy queen is good). My evaluation network also evaluates the four move checkmate as black favoured.</p>
  
Tags: <neural-networks><deep-learning><chess>
Title: Reason why chess neural network might not be training
UserDisplayName: Sam Liu
ThreadText: # Reason why chess neural network might not be training
  
  (Asked by: username_0 on 2022-01-16)
  
  I've been trying to use a Stockfish-like chess evaluation neural network for the past few weeks but to no avail. I wanted to get some other opinions about why my current methods haven't worked.
  Input: $8 \times 8 \times 12$ one-hot encoded board of pieces. I have around ~60 million unique examples. Evaluations are placed between -15 and 15 (anything greater or smaller is brought to 15 or -15)
  Output: Single number representing Stockfish evaluation.
  I've tried both fully connected and convolutional models and neither have worked very well. Here's some Tensorflow code that might give an idea of what the structures looked like:
  Convolutional model:
  import tensorflow as tf
  
  def block(filters, x, res=None):
    if res is not None:
      x = tf.keras.layers.concatenate([x, res])
    x = tf.keras.layers.Conv2D(filters=filters, kernel_size=3, padding='same', activation='relu', kernel_initializer='he_normal')(x)
    x = tf.keras.layers.BatchNormalization()(x)
    x = tf.keras.layers.MaxPooling2D(pool_size=2, strides=1, padding='same')(x)
    
    return x
  
  board = tf.keras.Input(shape=(8, 8, 12))
  conv1 = block(64, board)
  conv2 = block(128, conv1)
  conv3 = block(256, conv2)
  conv4 = block(512, conv3, conv3)
  conv5 = block(1024, conv4, conv2)
  conv6 = block(1024, conv5, conv1)
  conv7 = block(512, conv6)
  x = tf.keras.layers.Flatten()(conv7)
  x = tf.keras.layers.Dense(128, activation='relu')(x)
  x = tf.keras.layers.Dense(32, activation='relu')(x)
  x = tf.keras.layers.Dense(1)(x)
  
  Fully connected model:
  import tensorflow as tf
  
  board = tf.keras.Input(shape=(8, 8, 12))
  x = tf.keras.layers.Flatten()(board)
  x = tf.keras.layers.Dense(1024, activation='relu')(x)
  x = tf.keras.layers.Dense(1024, activation='relu')(x)
  x = tf.keras.layers.Dense(1024, activation='relu', kernel_regularizer='l2')(x)
  x = tf.keras.layers.Dropout(0.3)(x)
  x = tf.keras.layers.Dense(1024, activation='relu', kernel_regularizer='l2')(x)
  x = tf.keras.layers.Dropout(0.3)(x)
  x = tf.keras.layers.Dense(1024, activation='relu', kernel_regularizer='l2')(x)
  x = tf.keras.layers.Dropout(0.3)(x)
  x = tf.keras.layers.Dense(1024, activation='relu')(x)
  x = tf.keras.layers.Dense(1024, activation='relu')(x)
  x = tf.keras.layers.Dense(1024, activation='relu')(x)
  x = tf.keras.layers.Dense(256, activation='relu')(x)
  x = tf.keras.layers.Dense(1)(x)
  
  Just to add a few notes about what the current results are, they are decent (e.g. mean squared error of Stockfish evaluated cut off between -15 and 15 is about 7). The early game is also played fairly decently but the late game gets really bad (the engine can't evaluate that bishop takes enemy queen is good). My evaluation network also evaluates the four move checkmate as black favoured.
  
  **Tags:** neural-networks, deep-learning, chess
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2022-01-18. Score: 1
  
  It could be because there is simply not enough data for the late game. To make the model give more importance to the later stages of the game you can try to tweak the loss function such that it penalizes more for when there are fewer pieces on the board. ( This might give an idea on that: <https://medium.com/visionwizard/understanding-focal-loss-a-quick-read-b914422913e7>)
  
  One other thing you can do; given that my above-mentioned reasoning is correct, is to provide the number of pieces on the board as an input. I am not sure how you might go around doing that in the one-hot encoded scenario but might be worth looking into (You could append the number of pieces in the binary format to the flattened input). One argument against this is that the network should implicitly know how many pieces are on the board but yet it could still be useful to give it as an extra input.
  
  
  ---
  
  ### Answer by username_2 on 2023-10-04. Score: 1
  
  I tried hacking together several NN approaches for chess, in the context of [Chess Coding Challenge](https://github.com/SebLague/Chess-Challenge) (implement a chess bot in a relatively small amount of code). I also play online chess semi-regularly, so I have some understanding of the game. I found that in general the more accurate my network was with evaluation (and the more complex it was), the worse it played since it ended up in uncommon situations which it grossly mis-evaluated to its favor.
  
  I haven't done a detailed analysis, but I suspect that there two main factors here:
  
  * Lack of diversity in the training data, for example does your network correctly evaluate a starting position with queen odds?
  * The difficulty of evaluating positions which involve tactics
  
  Here are three similar positions, sorry it is always **black's turn to move**, since I encountered these when playing as black:
  
  [![chess-board-1](https://i.sstatic.net/y1VNa.png)](https://i.sstatic.net/y1VNa.png)
  
  Evaluation: -6.4
  
  [![chess-board-2](https://i.sstatic.net/4oxJr.png)](https://i.sstatic.net/4oxJr.png)
  
  Evaluation: -3.3
  
  [![chess-board-3](https://i.sstatic.net/GE90Z.png)](https://i.sstatic.net/GE90Z.png)
  
  Evaluation: -2.2
  
  I don't know a good solution on how to correctly evaluate tactics, except searching explicit steps in the game tree. Note the first position has a rather simple two-move tactic, but modern engines will do searches tens of moves deep.
  
  And regarding the first issue, I got better results by imposing strong regularization to the network's architecture and weights. For example a good starting point would be an evaluation function which resembles a [Piece-Square-Table](https://www.chessprogramming.org/Piece-Square_Tables).
  
  
  ---
  
  