Answers.Body: <p>What you are after is called a novelty, or better: <strong>novelties</strong>.</p>
  <p>That's formally (a) new move(s) in a certain position, but let's instead focus on an alternative, albeit slightly off, definition:</p>
  <blockquote>
  <p>A (prepared) novelty is a move that's in your book, and hopefully missing in your opponents.</p>
  </blockquote>
  <p>What's your book, alternatively your repertoire then?</p>
  <blockquote>
  <p>Your book is, again roughly, the sum of what you know by heart and are willing to play.</p>
  </blockquote>
  <p>How comes you are willing to play a certain line? - Well, you studied the line: you know the ins and outs of every reply your opponents could play; you understand <em>why</em> some replies are losing and <em>how</em> to punish those; <em>why</em> other replies are fine and <em>how</em> to proceed; which procedings can lead to <em>what</em> imbalances, pawn structures, endgames.</p>
  <p>And that's exactly where engines come in handy. Nota bene that before our age, when this was being done with brainpower alone, the concept was still the one mentioned above! You investigate, summarize, memorize.</p>
  <p>The investigation part is what you'd like to outsource now, and please feel free to do so! However, the follow-ups are still heavy workloads in summarizing (sort of translating into terms understandable to humans, the <em>whys</em> from above) and memorizing (sort of the <em>hows</em>). Thus, great idea, but the flip side is just that oh so many lines will take massive workloads to fully comprehend.</p>
  <p>In essence:
  The difference between humans and computers is substantial enough to exactly build a repertoire upon it - if you're willing to do the comprehensional workloads.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-10T20:11:48.727Z
Answers.Text: I wonder if people miss building strength by exploring novelties on their own.  I think the best approach would be to try out a bunch of moves without the engine first, then test out the idea using engines. That way, the heavy lifting done by engines doesn't detract from an aha moment.
Answers.UserDisplayName: DdogBoss
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-10T09:05:24.707Z
Answers.LastActivityDate: 2022-06-10T09:05:24.707Z
Answers.UserDisplayName: Peter Fischer
Answers.Body: <p>I will not be able to answer your question fully but can guide you here. There is a field of Machine Learning called Reinforcement Learning which deals with this exactly. There has been an example case of a computer playing checkers against itself for a long time and ultimately learning to play better than the programmer itself. I suppose that similar RL Models would be existing for Chess too, you could look for it and it might help you.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-11T17:40:08.890Z
Answers.LastActivityDate: 2022-06-11T18:01:44.183Z
Answers.LastEditDate: 2022-06-11T18:01:44.183Z
Answers.UserDisplayName: user32658
Answers.Body: <p>If you're concerned about whether an opening is objectively good, then using computer games is a good approach. Computers don't play perfectly, but they are by far our best option. You could also check out correspondence games from ICCF, which involves people sometimes having days to figure out each move, and they can use an engine. These games, as well as TCEC games, are some of the strongest games available.</p>
  <p>However, if you're interested in how good an opening is <em>practically</em> (i.e., for human OTB play), then it depends on what you mean. Personally, I'd think an opening is practically good if it is fairly sound objectively, but also something I'd be comfortable playing in an OTB game.</p>
  <p>Both of these requirements are somewhat subjective - e.g., what is &quot;fairly sound objectively&quot;? Some amateurs might say anything between 0.00 and -1.00 is fine for their Black opening, while super GMs may not be satisfied with anything worse than -0.20. Even more subjective is whether an opening is comfortable for you, since this comes down to playing styles.</p>
  <p>To that end, it could be worthwhile to have an engine play against itself to figure out how good an opening is objectively. Although, there will likely already be enough games out there (TCEC, correspondence, high level GM games). You can also see how well an opening scores amongst all players in big databases, such as TWIC or the Mega Database. This could give an indication of how easy it is to play an opening, for people who aren't professionals or anything.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-10T15:09:48.817Z
Answers.Text: (+1) for raising the **Practicality** issue
Answers.UserDisplayName: SecretAgentMan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-10T20:12:53.127Z
Answers.Text: Super GMs still go for openings that swing up to -1.00. The Benoni, Alekhine are played at the top level albeit infrequently.
Answers.UserDisplayName: DdogBoss
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-10T23:28:14.147Z
Answers.Text: @DdogBoss True, but as you said infrequently, mainly as a surprise weapon or in fast time controls.
Answers.UserDisplayName: Inertial Ignorance
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-10T09:00:43.283Z
Answers.LastActivityDate: 2022-06-10T09:00:43.283Z
Answers.UserDisplayName: Inertial Ignorance
Body: <p>Recently, I got into a brief argument with a colleague when I mentioned that I wanted to use an engine playing against itself to generate a larger sample size for a rare sideline.</p>
  <p>His argument was that human chess and computer chess are two different things, which made sense. Humans don't scan for every possible legal move, and don't use brute calculation like computers do.</p>
  <p>However, there may not be a time when masters decide to play a given line, meaning that rare lines would have fewer games played. There would be no other way to build a larger dataset other than playing games yourself or waiting for other people to play it.</p>
  <p>Another factor to consider is how seriously people take TCEC openings.</p>
  <p><strong>Is the difference between humans and computers substantial enough to dismiss opening testing with an engine playing against itself? Would the use of computer games be a flawed sample size for determining the validity of a move?</strong></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-06-10T03:59:48.353Z
Comments.Text: When you use the word 'viable', viable for what? Human play?
Comments.UserDisplayName: Allure
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-06-10T04:12:26.013Z
Comments.Text: @Allure I am trying to build a database of games surrounding a rare line. Human games capture the complexity of a move with errors. Computers play perfect chess, and it is unreasonable for a human to play at the level of a computer. Would it be correct to build a database of computer games, a database of human games, or a hybrid? Viable in which database would most accurately represent move viability in human play.
Comments.UserDisplayName: DdogBoss
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-06-10T08:24:57.883Z
Comments.Text: How would you prevent getting the same game over and over again? I guess a better approach would be to use different engines. Also, LeelaChess has a [Temperature parameter](https://lczero.org/dev/wiki/lc0-options/) that allows it some randomness, and you can tune how much randomness and for how long you want it.
Comments.UserDisplayName: emdio
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-06-10T09:00:01.713Z
Comments.Text: @emdio One way is time control. One thing limiting the engine's search is a time limit. Even if, in different runs, the same engine faces the same position, the respective searches (curtailed by time) will be of different portions of the search space, and in one run it might spot a good move it didn't see in the other run. Or in one run it might choose a move because it didn't see a refutation, whereas in the other run it saw the refutation and rejected the move.
Comments.UserDisplayName: Rosie F
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-06-10T09:25:53.450Z
Comments.Text: @RosieF that's interesting. Besides time/depth variations, maybe you can just try using several engines. I guess many engines in the top 20 are opensource or free. Also, you can try different versions of the same Engine (v. g. Stockfish 11 to 15), And then again, in the last versions of Stockfish you can use the "classical" and the NNUE evaluation.
Comments.UserDisplayName: emdio
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-06-10T15:06:21.727Z
Comments.Text: "Computers play perfect chess": absolutely not! They just play much better than humans.
Comments.UserDisplayName: TonyK
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-06-10T20:15:39.713Z
Comments.Text: @emdio This is where a human component would come into play. With a novelty, there may be several lines. You would nudge the engine towards a different position by changing up the line, but it would still be the same novelty. Using a faster time control gets some randomness because engines won't have time to process every move to full depth.
Comments.UserDisplayName: DdogBoss
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-06-11T05:25:46.143Z
Comments.Text: Edited for grammar.
Comments.UserDisplayName: Savage47
Tags: <opening><engines><statistics><database>
Title: Is testing an opening by having a computer play against itself viable?
ThreadText: # Is testing an opening by having a computer play against itself viable?
  
  (Asked by: username_0 on 2022-06-10)
  
  Recently, I got into a brief argument with a colleague when I mentioned that I wanted to use an engine playing against itself to generate a larger sample size for a rare sideline.
  
  His argument was that human chess and computer chess are two different things, which made sense. Humans don't scan for every possible legal move, and don't use brute calculation like computers do.
  
  However, there may not be a time when masters decide to play a given line, meaning that rare lines would have fewer games played. There would be no other way to build a larger dataset other than playing games yourself or waiting for other people to play it.
  
  Another factor to consider is how seriously people take TCEC openings.
  
  **Is the difference between humans and computers substantial enough to dismiss opening testing with an engine playing against itself? Would the use of computer games be a flawed sample size for determining the validity of a move?**
  
  **Tags:** opening, engines, statistics, database
  
  > **Comments:**
  > **username_1**: When you use the word 'viable', viable for what? Human play?  
  > **username_0**: @username_1 I am trying to build a database of games surrounding a rare line. Human games capture the complexity of a move with errors. Computers play perfect chess, and it is unreasonable for a human to play at the level of a computer. Would it be correct to build a database of computer games, a database of human games, or a hybrid? Viable in which database would most accurately represent move viability in human play.  
  > **username_2**: How would you prevent getting the same game over and over again? I guess a better approach would be to use different engines. Also, LeelaChess has a [Temperature parameter](https://lczero.org/dev/wiki/lc0-options/) that allows it some randomness, and you can tune how much randomness and for how long you want it.  
  > **username_3**: @username_2 One way is time control. One thing limiting the engine's search is a time limit. Even if, in different runs, the same engine faces the same position, the respective searches (curtailed by time) will be of different portions of the search space, and in one run it might spot a good move it didn't see in the other run. Or in one run it might choose a move because it didn't see a refutation, whereas in the other run it saw the refutation and rejected the move.  
  > **username_2**: @RosieF that's interesting. Besides time/depth variations, maybe you can just try using several engines. I guess many engines in the top 20 are opensource or free. Also, you can try different versions of the same Engine (v. g. Stockfish 11 to 15), And then again, in the last versions of Stockfish you can use the "classical" and the NNUE evaluation.  
  > **username_4**: "Computers play perfect chess": absolutely not! They just play much better than humans.  
  > **username_0**: @username_2 This is where a human component would come into play. With a novelty, there may be several lines. You would nudge the engine towards a different position by changing up the line, but it would still be the same novelty. Using a faster time control gets some randomness because engines won't have time to process every move to full depth.  
  > **username_5**: Edited for grammar.  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_8 on 2022-06-10. Score: 11 | Accepted answer
  
  If you're concerned about whether an opening is objectively good, then using computer games is a good approach. Computers don't play perfectly, but they are by far our best option. You could also check out correspondence games from ICCF, which involves people sometimes having days to figure out each move, and they can use an engine. These games, as well as TCEC games, are some of the strongest games available.
  
  However, if you're interested in how good an opening is *practically* (i.e., for human OTB play), then it depends on what you mean. Personally, I'd think an opening is practically good if it is fairly sound objectively, but also something I'd be comfortable playing in an OTB game.
  
  Both of these requirements are somewhat subjective - e.g., what is "fairly sound objectively"? Some amateurs might say anything between 0.00 and -1.00 is fine for their Black opening, while super GMs may not be satisfied with anything worse than -0.20. Even more subjective is whether an opening is comfortable for you, since this comes down to playing styles.
  
  To that end, it could be worthwhile to have an engine play against itself to figure out how good an opening is objectively. Although, there will likely already be enough games out there (TCEC, correspondence, high level GM games). You can also see how well an opening scores amongst all players in big databases, such as TWIC or the Mega Database. This could give an indication of how easy it is to play an opening, for people who aren't professionals or anything.
  
  > **Comments:**
  > **username_9**: (+1) for raising the **Practicality** issue  
  > **username_0**: Super GMs still go for openings that swing up to -1.00. The Benoni, Alekhine are played at the top level albeit infrequently.  
  > **username_8**: @username_0 True, but as you said infrequently, mainly as a surprise weapon or in fast time controls.  
  
  ---
  
  ### Answer by username_6 on 2022-06-10. Score: 4
  
  What you are after is called a novelty, or better: **novelties**.
  
  That's formally (a) new move(s) in a certain position, but let's instead focus on an alternative, albeit slightly off, definition:
  
  > A (prepared) novelty is a move that's in your book, and hopefully missing in your opponents.
  
  What's your book, alternatively your repertoire then?
  
  > Your book is, again roughly, the sum of what you know by heart and are willing to play.
  
  How comes you are willing to play a certain line? - Well, you studied the line: you know the ins and outs of every reply your opponents could play; you understand *why* some replies are losing and *how* to punish those; *why* other replies are fine and *how* to proceed; which procedings can lead to *what* imbalances, pawn structures, endgames.
  
  And that's exactly where engines come in handy. Nota bene that before our age, when this was being done with brainpower alone, the concept was still the one mentioned above! You investigate, summarize, memorize.
  
  The investigation part is what you'd like to outsource now, and please feel free to do so! However, the follow-ups are still heavy workloads in summarizing (sort of translating into terms understandable to humans, the *whys* from above) and memorizing (sort of the *hows*). Thus, great idea, but the flip side is just that oh so many lines will take massive workloads to fully comprehend.
  
  In essence:
  The difference between humans and computers is substantial enough to exactly build a repertoire upon it - if you're willing to do the comprehensional workloads.
  
  > **Comments:**
  > **username_0**: I wonder if people miss building strength by exploring novelties on their own.  I think the best approach would be to try out a bunch of moves without the engine first, then test out the idea using engines. That way, the heavy lifting done by engines doesn't detract from an aha moment.  
  
  ---
  
  ### Answer by username_7 on 2022-06-11. Score: 1
  
  I will not be able to answer your question fully but can guide you here. There is a field of Machine Learning called Reinforcement Learning which deals with this exactly. There has been an example case of a computer playing checkers against itself for a long time and ultimately learning to play better than the programmer itself. I suppose that similar RL Models would be existing for Chess too, you could look for it and it might help you.
  
  
  ---
  
  