Answers.Body: <p>Learning to play better chess by itself is a part of reinforcement machine learning (temporal difference learning). A simple google will give you some academic papers. My recommended papers (that I've read myself):</p>
  
  <ul>
  <li><a href="https://chessprogramming.wikispaces.com/KnightCap" rel="nofollow noreferrer">KnightCap</a></li>
  <li><a href="https://chessprogramming.wikispaces.com/NeuroChess" rel="nofollow noreferrer">NeuroChess</a></li>
  <li><a href="http://arxiv.org/abs/1509.01549" rel="nofollow noreferrer">Giraffe</a></li>
  </ul>
  
  <p>While the programs are different. They share the similar idea:</p>
  
  <p>The idea is like this:</p>
  
  <ol>
  <li>Come up with an evaluation function that you think appropriate</li>
  <li>Implement a search algorithm (mini-max, nega-max etc)</li>
  <li>Initialise the weights in the evaluation function. The weights would be your <code>features</code></li>
  <li>Play a game against itself</li>
  <li>Record the result of the game and compute the error.</li>
  <li>Apply the following TD-Leaf equation to change the weights</li>
  </ol>
  
  <p><a href="https://i.sstatic.net/UPXEl.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/UPXEl.png" alt="enter image description here"></a></p>
  
  <ol start="7">
  <li>Play more games until you believe your values have converged</li>
  </ol>
  
  <p>The equation updates the weight vector by the gradient, scaled by a constant (learning rate). The errors would be the temporal difference (d(t)) in the equation. They would need to be added for all moves until the end of game, scaled by the distance from <code>t</code>. Please google <code>TD-Leaf</code> if you want to know more.</p>
  
  <p>While there have been some serious attempts at applying the TD-Leaf algorithm, none of which was strong enough to compete with Komodo and Stockfish. I think that's because:</p>
  
  <ul>
  <li>Lack of determination. The authors wrote their TD-Leaf chess engines for their research. Once that was done, they moved on to something else, something much more <strong>profitable</strong> for them. For example, the NeuroChess program was written by a very well-known AI export. He works for Google and I think Google has more challenging task for him.</li>
  <li>Weak search algorithm</li>
  <li>Not challenging anymore. A human tuning chess engine can easily defeat the best grandmaster. Why would we waste time and resources to further improve it? Why not spend the money on <code>Go</code>, where the best human player has yet to be beaten?</li>
  </ul>
  
  <p>Some remarks:</p>
  
  <ul>
  <li>While it's possible to train a model from random weights, we don't want to do this because it would lead to extremely long training times. We can give starting values with some human-knowledge, or bootstrapping (read the Giraffe paper for details).</li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-02-27T06:17:42.823Z
Answers.LastActivityDate: 2016-02-27T08:33:59.260Z
Answers.LastEditDate: 2016-02-27T08:33:59.260Z
Answers.UserDisplayName: SmallChess
Body: <p>Are there any chess engines that can adapt and 'learn' how to play better with an opponent? One with certain system parameters that can be adjusted while playing so the computer can 'respond' better to a human player?</p>
  
Tags: <engines><learning>
Title: Chess engines as similar to A.I
ThreadText: # Chess engines as similar to A.I
  
  (Asked by: username_0 on 2016-02-27)
  
  Are there any chess engines that can adapt and 'learn' how to play better with an opponent? One with certain system parameters that can be adjusted while playing so the computer can 'respond' better to a human player?
  
  **Tags:** engines, learning
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2016-02-27. Score: 4 | Accepted answer
  
  Learning to play better chess by itself is a part of reinforcement machine learning (temporal difference learning). A simple google will give you some academic papers. My recommended papers (that I've read myself):
  
  * [KnightCap](https://chessprogramming.wikispaces.com/KnightCap)
  * [NeuroChess](https://chessprogramming.wikispaces.com/NeuroChess)
  * [Giraffe](http://arxiv.org/abs/1509.01549)
  
  While the programs are different. They share the similar idea:
  
  The idea is like this:
  
  1. Come up with an evaluation function that you think appropriate
  2. Implement a search algorithm (mini-max, nega-max etc)
  3. Initialise the weights in the evaluation function. The weights would be your `features`
  4. Play a game against itself
  5. Record the result of the game and compute the error.
  6. Apply the following TD-Leaf equation to change the weights
  
  [![enter image description here](https://i.sstatic.net/UPXEl.png)](https://i.sstatic.net/UPXEl.png)
  
  7. Play more games until you believe your values have converged
  
  The equation updates the weight vector by the gradient, scaled by a constant (learning rate). The errors would be the temporal difference (d(t)) in the equation. They would need to be added for all moves until the end of game, scaled by the distance from `t`. Please google `TD-Leaf` if you want to know more.
  
  While there have been some serious attempts at applying the TD-Leaf algorithm, none of which was strong enough to compete with Komodo and Stockfish. I think that's because:
  
  * Lack of determination. The authors wrote their TD-Leaf chess engines for their research. Once that was done, they moved on to something else, something much more **profitable** for them. For example, the NeuroChess program was written by a very well-known AI export. He works for Google and I think Google has more challenging task for him.
  * Weak search algorithm
  * Not challenging anymore. A human tuning chess engine can easily defeat the best grandmaster. Why would we waste time and resources to further improve it? Why not spend the money on `Go`, where the best human player has yet to be beaten?
  
  Some remarks:
  
  * While it's possible to train a model from random weights, we don't want to do this because it would lead to extremely long training times. We can give starting values with some human-knowledge, or bootstrapping (read the Giraffe paper for details).
  
  
  ---
  
  