Answers.Body: <p>In Chess, you use SPSA and CLOP. Those are the algorithms used by Stockfish in Fishtest to tune its parameter. The algorithm worked so well that it made Stockfish be the strongest engine in the world.</p>
  
  <p>Please note that the algorithms are very complicated mathematically. One could write a PHD thesis on this topic. In SPSA, you start from a best guess from the value. Then play games, use those games to derive a better gradient for the next series of parameters.</p>
  
  <p>The maths is very hard, but it all boils down to some statistics - you play games, and use those games to derive a distribution to update the parameters you want to tune.</p>
  
  <p>Please read:</p>
  
  <p><a href="http://talkchess.com/forum/viewtopic.php?start=0&amp;t=50823&amp;topic_view=flat" rel="nofollow">http://talkchess.com/forum/viewtopic.php?start=0&amp;t=50823&amp;topic_view=flat</a></p>
  
  <p>and </p>
  
  <p><a href="https://groups.google.com/forum/#!searchin/fishcooking/SPSA/fishcooking/WNrxeXAJ6VI/ZkCnRv4I_qEJ" rel="nofollow">https://groups.google.com/forum/#!searchin/fishcooking/SPSA/fishcooking/WNrxeXAJ6VI/ZkCnRv4I_qEJ</a></p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2014-08-27T03:59:30.443Z
Answers.LastActivityDate: 2014-08-27T03:59:30.443Z
Answers.UserDisplayName: SmallChess
Answers.Body: <p>I might approach this via <a href="http://en.wikipedia.org/wiki/Genetic_algorithm" rel="nofollow">genetic algorithm</a>.  </p>
  
  <ol>
  <li>Each param in the gene represents one weight you use for tuning.  </li>
  <li>Have a fairly large test suite of positions + moves made by grandmasters (randomly over many middle game positions).</li>
  <li>Create a random initial population of genes to be used in competition.  You can include any feature weights you already believe to be good as well.</li>
  <li>For each gene, for each position, do a shallow search based on how much training time you want to take.</li>
  <li>For every position where your program produces the same move as a grandmaster, add one to a total associated with a gene.  This total is what the GA should try to maximize.</li>
  <li>Random pairing for mating rights competition based on above score, and breed the results using standard GA techniques.</li>
  <li>Run until convergence.</li>
  </ol>
  
  <p>Whatever final weight you get becomes your feature weight.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2014-08-27T00:51:53.107Z
Answers.Text: +1, and I hope someone will perform the following offshoot of this idea: for a handful of top players, craft an evaluation function via the described method *using only the moves of a single player* in order to test gene fitness. Then, build engines that are identical except for the individually-tuned evaluation functions, and have them duke it out. It may be the closest we could ever come to, say, answering the age-old question of who would have won the Fischer-Karpov match that never happened :-)  ... (cont'd)
Answers.UserDisplayName: ETD
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2014-08-27T00:52:26.643Z
Answers.Text: (cont'd) ... I don't seriously think we could draw historical conclusions, of course, but it would be somewhat interesting in and of itself if one player's engine version tended to outperform another's empirically.
Answers.UserDisplayName: ETD
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2014-08-27T04:00:55.763Z
Answers.Text: -1 While it's not wrong, it makes no attempt to how the algorithm can be applied in chess. Stockfish has ALREADY APPLIED a SPSA algorithm. Read my answer.
Answers.UserDisplayName: SmallChess
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2014-08-27T17:35:42.823Z
Answers.Text: what are you talking about?  I explicitly laid out how to apply it to chess.  I suspect you didn't read the steps.
  
  There are many papers in scholastic literature that follow a similar approach btw.  Just because Stockfish has chosen a different approach to numerical optimization of heuristic coefficients doesn't mean this answer should be down voted.
Answers.UserDisplayName: tbischel
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2014-08-27T20:59:06.533Z
Answers.Text: @StudentT, tbischel's entire answer is precisely a detailed recipe for applying the idea in question to chess. Given that fact, and that you say yourself this answer is in no way wrong, your downvote is rather puzzling.
Answers.UserDisplayName: ETD
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2014-10-28T16:37:14.390Z
Answers.Text: I can help explain: @StudentT's answer was more direct. It gives the exact algorithms and mathematics to use. tbischel's answer here is more indirectly. It gives the idea of what to do, but not how to do it. Both are valuable answers to me.
Answers.UserDisplayName: Xonatron
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2014-08-26T23:55:30.677Z
Answers.LastActivityDate: 2014-08-26T23:55:30.677Z
Answers.UserDisplayName: tbischel
Body: <p>I am developing a chess program. And have made use of an alpha beta algorithm and a static evaluation function. I have successfully implemented both but I want to improve the evaluation function by automatically tuning the weights assigned to its features. At this point am totally confused about the policy suitable for updating the weights of the function. One policy I have in mind is to check whether a move is good or bad before updating weights but I don't really know how to implement it. Thus I need ideas and pseudo code please.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2014-08-25T16:04:17.060Z
Comments.Text: This question is probably more suitable for the [Programmers SE](http://programmers.stackexchange.com/).
Comments.UserDisplayName: Henry Keiter
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2014-08-25T19:15:41.787Z
Comments.Text: This question is more suitable for [Stack Overflow SE](http://stackoverflow.com/).
Comments.UserDisplayName: AlwaysLearningNewStuff
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2014-08-25T19:37:38.110Z
Comments.Text: You might want to look at [chess programming wiki](https://chessprogramming.wikispaces.com/Engines).
Comments.UserDisplayName: Pete Becker
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2014-08-26T11:21:45.617Z
Comments.Text: This is a really hard problem, I don't think any strong chess engines update the weights automatically.
Comments.UserDisplayName: RemcoGerlich
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2014-08-26T23:42:32.023Z
Comments.Text: Deep Blue self trained its evaluation function based on how well it could match the moves of GMs... so this is a common strategy.
Comments.UserDisplayName: tbischel
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2014-09-01T07:32:28.973Z
Comments.Text: This is more of a AI question (2-player adverserial) than a chess one
Comments.UserDisplayName: prusswan
Tags: <learning><software><evaluation>
Title: understanding evaluation function
ThreadText: # understanding evaluation function
  
  (Asked by: username_0 on 2014-08-25)
  
  I am developing a chess program. And have made use of an alpha beta algorithm and a static evaluation function. I have successfully implemented both but I want to improve the evaluation function by automatically tuning the weights assigned to its features. At this point am totally confused about the policy suitable for updating the weights of the function. One policy I have in mind is to check whether a move is good or bad before updating weights but I don't really know how to implement it. Thus I need ideas and pseudo code please.
  
  **Tags:** learning, software, evaluation
  
  > **Comments:**
  > **username_1**: This question is probably more suitable for the [Programmers SE](http://programmers.stackexchange.com/).  
  > **username_2**: This question is more suitable for [Stack Overflow SE](http://stackoverflow.com/).  
  > **username_3**: You might want to look at [chess programming wiki](https://chessprogramming.wikispaces.com/Engines).  
  > **username_4**: This is a really hard problem, I don't think any strong chess engines update the weights automatically.  
  > **username_5**: Deep Blue self trained its evaluation function based on how well it could match the moves of GMs... so this is a common strategy.  
  > **username_6**: This is more of a AI question (2-player adverserial) than a chess one  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_7 on 2014-08-27. Score: 2
  
  In Chess, you use SPSA and CLOP. Those are the algorithms used by Stockfish in Fishtest to tune its parameter. The algorithm worked so well that it made Stockfish be the strongest engine in the world.
  
  Please note that the algorithms are very complicated mathematically. One could write a PHD thesis on this topic. In SPSA, you start from a best guess from the value. Then play games, use those games to derive a better gradient for the next series of parameters.
  
  The maths is very hard, but it all boils down to some statistics - you play games, and use those games to derive a distribution to update the parameters you want to tune.
  
  Please read:
  
  <http://talkchess.com/forum/viewtopic.php?start=0&t=50823&topic_view=flat>
  
  and
  
  <https://groups.google.com/forum/#!searchin/fishcooking/SPSA/fishcooking/WNrxeXAJ6VI/ZkCnRv4I_qEJ>
  
  
  ---
  
  ### Answer by username_5 on 2014-08-26. Score: 1
  
  I might approach this via [genetic algorithm](http://en.wikipedia.org/wiki/Genetic_algorithm).
  
  1. Each param in the gene represents one weight you use for tuning.
  2. Have a fairly large test suite of positions + moves made by grandmasters (randomly over many middle game positions).
  3. Create a random initial population of genes to be used in competition. You can include any feature weights you already believe to be good as well.
  4. For each gene, for each position, do a shallow search based on how much training time you want to take.
  5. For every position where your program produces the same move as a grandmaster, add one to a total associated with a gene. This total is what the GA should try to maximize.
  6. Random pairing for mating rights competition based on above score, and breed the results using standard GA techniques.
  7. Run until convergence.
  
  Whatever final weight you get becomes your feature weight.
  
  > **Comments:**
  > **username_8**: +1, and I hope someone will perform the following offshoot of this idea: for a handful of top players, craft an evaluation function via the described method *using only the moves of a single player* in order to test gene fitness. Then, build engines that are identical except for the individually-tuned evaluation functions, and have them duke it out. It may be the closest we could ever come to, say, answering the age-old question of who would have won the Fischer-Karpov match that never happened :-)  ... (cont'd)  
  > **username_8**: (cont'd) ... I don't seriously think we could draw historical conclusions, of course, but it would be somewhat interesting in and of itself if one player's engine version tended to outperform another's empirically.  
  > **username_7**: -1 While it's not wrong, it makes no attempt to how the algorithm can be applied in chess. Stockfish has ALREADY APPLIED a SPSA algorithm. Read my answer.  
  > **username_5**: what are you talking about?  I explicitly laid out how to apply it to chess.  I suspect you didn't read the steps.  There are many papers in scholastic literature that follow a similar approach btw.  Just because Stockfish has chosen a different approach to numerical optimization of heuristic coefficients doesn't mean this answer should be down voted.  
  > **username_8**: @StudentT, username_5's entire answer is precisely a detailed recipe for applying the idea in question to chess. Given that fact, and that you say yourself this answer is in no way wrong, your downvote is rather puzzling.  
  > **username_9**: I can help explain: @StudentT's answer was more direct. It gives the exact algorithms and mathematics to use. username_5's answer here is more indirectly. It gives the idea of what to do, but not how to do it. Both are valuable answers to me.  
  
  ---
  
  