Answers.Body: <p>This happens often enough in engine tournaments that viewers have coined the terms &quot;boom&quot; and &quot;moob&quot; (for &quot;eval explodes away from 0&quot; and &quot;eval implodes towards zero&quot; respectively).</p>
  <p>Here's the <a href="https://tcec-chess.com/#div=vsob19&amp;game=38&amp;season=19" rel="nofollow noreferrer">latest example of a boom</a> from TCEC.</p>
  <pre><code>[FEN &quot;8/kp6/6B1/p3p3/3r3P/q5P1/2Q2PK1/8 w - - 0 59&quot;]
  </code></pre>
  <p>Leela, playing White, boomed from -0.46 the move before to -1.91 after playing 59. h5. That means this position (or one of the moves slightly prior) very likely meets your criteria, if you let Leela think for a sufficiently long time.</p>
  <p>You can also see from the principal variations of both engines that Leela did not anticipate Komodo's move on move 58 - Leela predicted 58...Qd5+, but Komodo played 58...Qxa3.</p>
  <p>If you're looking only for booms of more than +3, off the top of my head, <a href="https://en.wikipedia.org/wiki/TCEC_Season_14#Game_85" rel="nofollow noreferrer">this</a> was another memorable boom.</p>
  <pre><code>[FEN &quot;7Q/8/P3n1p1/2p3k1/6b1/5pb1/8/5K2 w - - 1 59&quot;]
  </code></pre>
  <p>Leela, playing White, evaluated the position after 59.a7 as -0.06, then boomed to -14.28 after 59...Bf5 (Leela predicted 59...Nc7).</p>
  <p><strong>Edited to add</strong>: here's <a href="https://en.wikipedia.org/wiki/TCEC_Season_19#Game_9" rel="nofollow noreferrer">an example of a Stockfish moob from -1.38 to 0.00</a> after thinking for 4 minutes on TCEC hardware.</p>
  <pre><code>[FEN &quot;r4k1r/p1pb2bp/2pp1nq1/4P3/3P3Q/2N5/PPP3PP/R4RK1 w - - 3 16&quot;]
  
  1.Rae1 Bf5
  </code></pre>
  <p>Stockfish had the Black pieces and its eval dropped after ...Bf5. <a href="https://tcec-chess.com/#div=sf&amp;game=9&amp;season=19" rel="nofollow noreferrer">Link to game</a> with both engines' PVs if you want to see those.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-27T16:36:45.210Z
Answers.Text: Yes, it's booms and moobs that I am interested in. (Thanks, by the way. I was not aware of these "terms".) How long did it take Leela to boom in the second one though? The latest Stockfish on my laptop finds mate in 22 in less than a minute. Do you have any examples where a boom (or moob) occurs after 2-3 minutes of evaluation?
Answers.UserDisplayName: the_fox
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-27T18:21:38.753Z
Answers.Text: @the_fox the second one used a (by now) very old Leela net, which had very noticeable weaknesses. That's why Stockfish finds the mate quickly and the Leela missed it completely. The time taken for an engine to boom / moob can be quite a while because that time taken is completely dependent on the hardware you're running the program on. Nonetheless, I'm adding another example, this time of a Stockfish moob which took several minutes on TCEC hardware (i.e. on your hardware, Stockfish will probably take longer to moob).
Answers.UserDisplayName: Allure
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-27T12:17:44.270Z
Answers.LastActivityDate: 2020-11-27T18:26:54.570Z
Answers.LastEditDate: 2020-11-27T18:26:54.570Z
Answers.UserDisplayName: Allure
Answers.Body: <p>This is a complicated position I had in a correspondence game. White to play and win. The engine evaluation rises and falls, occasionally hitting 0 again. Then it stays at roughly 1.5 for a long while until finally finding the breakthrough about 20 minutes in. Unsurprisingly, considering the full line is pretty insane.</p>
  <pre><code>[FEN &quot;r5r1/1np3bk/1p1p4/3P4/bpP2PPp/P3N2P/1q1BQ2K/4R1R1 w - - 0 33&quot;]
  
  1. g5 Rge8 2. Qh5+ Kg8 3. Rg2 bxa3 4. Nf5 Rxe1 5. Bxe1 Bc2 6. Nh6+ Kh8 7. Ng4+ Kg8 8. Nf6+ Kf8 9. Nd7+ Ke7 10. Qg4 Nc5 11. g6 Rh8 12. Re2+ Kd8 13. Nb8 Ba4 14. Nc6+ Bxc6 15. dxc6 Qf6 16. Bf2 Re8 17. Ra2 Qe6 18. Bxh4+ Kc8 19. Re2 Kb8 20. Rxe6 Rxe6 21. Qg2 Re3 22. f5 Be5+ 23. Kg1 Rb3 24. Qa2 Bd4+ 25. Kf1 Rf3+ 26. Kg2 Rb3 27. f6 Rb2+ 28. Kf3 Rxa2 29. g7 Rg2 30. Kxg2 a2 31. g8=Q+ Ka7 32. f7 a1=Q 33. f8=Q Qg1+ 34. Kf3 Qd1+ 35. Kf4 Ne6+ 36. Qxe6 Qf1+ 37. Ke4 Qxf8 38. Qe7 Qxe7+ 39. Bxe7
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-26T01:31:01.180Z
Answers.Text: You may have to reset the move counter.. Start at 1. Idk if that's the only way but it seemed to work.
Answers.UserDisplayName: mowwwalker
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-26T01:42:43.187Z
Answers.Text: I should have been more careful in the phrasing of the problem. The move g5 doesn't quite fit the bill of what I am looking for because it is one of the original moves suggested. The evaluation does change by more than 3.0 if you let the engine think quite a bit, but what I am really looking for is a drastic change in the evaluation suggested by a move not previously considered.
Answers.UserDisplayName: the_fox
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-26T02:54:04.507Z
Answers.Text: @mowwwalker that worked indeed, thanks. A bit annoying but what can you do.
Answers.UserDisplayName: koedem
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-26T02:55:00.430Z
Answers.Text: @the_fox I see. There certainly are examples for that behavior too, however I don't know a fitting position off the top of my head. Let's see if I remember one later.
Answers.UserDisplayName: koedem
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-26T09:59:13.830Z
Answers.Text: Basically, I am looking for "aha" moments from the engine.
Answers.UserDisplayName: the_fox
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-26T11:22:15.263Z
Answers.Text: If you fast-forward to the point where the line suggested by the original evaluation and the updated evaluation differ, would that give you the ah-ha moment? Presumably the engine spent a while on that first line before considering the second.
Answers.UserDisplayName: Ganesh Sittampalam
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-26T12:43:11.390Z
Answers.Text: @GS-ApologisetoMonica Good point, tried that now. Unfortunately while many positions on the way take a bit of time to switch the move, none of them take very long (so the accumulated time presumably simply contains the time of all switches along the way plus alternatives which also have to be calculated of course).
  
  A better way to find such a position is probably one of those short puzzles that Stockfish doesn't find and then wait until, well, it finds it. But the time it takes for something like that might vary a lot / be rather long.
Answers.UserDisplayName: koedem
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-26T15:58:51.187Z
Answers.Text: @GS-ApologisetoMonica It would, yes. But how often is it that the line the engine has settled on within a minute differs drastically in evaluation from the line it thinks best within half an hour? Assuming the second line is different from the first, that is. I am guessing that this doesn't happen very often. Of particular interest would be the situation where a position is evaluated as nearly equal at the beginning, only to find a winning move (much) later on.
Answers.UserDisplayName: the_fox
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-26T00:40:46.027Z
Answers.LastActivityDate: 2020-11-26T02:53:13.127Z
Answers.LastEditDate: 2020-11-26T02:53:13.127Z
Answers.UserDisplayName: koedem
Answers.Body: <p>Adding another answer since it meets every requirement by the OP and I believe they'll be interested.</p>
  <pre><code>[FEN &quot;4nr2/5ppk/pn3q2/3ppNRP/1pr1P3/1P3P2/1KP3Q1/R7 w - - 0 42&quot;]
  </code></pre>
  <p><a href="https://tcec-chess.com/#div=sf&amp;game=99&amp;season=20" rel="nofollow noreferrer">Game 99 of the Season 20 TCEC Superfinal</a></p>
  <p>Leela had +0.03 eval with this principal variation:</p>
  <pre><code>41....d5 42. bxc4 Nxc4+ 43. Kb1 dxe4 44. fxe4 Na3+ 45. Ka2 Qe6+ 46. Kb2 Nc4+ 47. Kb1 Rg8 48. Rg6 Qd7 49. Ka2 Qa4+ 50. Kb1 Qd7
  </code></pre>
  <p>And it boomed to +10.59 after 42. Rg6 and 3 minutes 27s thought.</p>
  <p>Of course, it is TCEC hardware - their Leela ran at 92.1 knps. It's extremely unlikely OP (or anyone) can get anywhere near this speed. With weaker hardware, Leela's eval will be different.</p>
  <p><em>However</em> it is worth emphasizing that OP's actual question (about the engine changing its mind during the search) is not so clear, because Leela could very well suffer from tunnel vision and completely miss 42. Rg6. That's not to say that Leela never changes its mind, it does, but it's not as simple as &quot;run Leela for 3 minutes and it will boom&quot;. That's especially the case since what Leela sees in 3 minutes is heavily dependent on what hardware you run Leela on.</p>
  <p>Here's <a href="https://matthewsadler.me.uk/the-middlegame/anatomy-of-a-blunder/" rel="nofollow noreferrer">an article</a> by GM Matthew Sadler on a different blunder by Leela in the TCEC Season 20 superfinal illustrating the difficulties with the question, and possibly providing another interesting position. To quote:</p>
  <blockquote>
  <p>Leave “my Leela” long enough (16.2M nodes searched) and its evaluation plummets: it sees that 79…Nb5 loses to the 80.Nf5 sacrifice  and its assessment of its new top move (79…Bb5) is just 27.9% expected score (1 win, 3 draws and 6 losses). A big fall!</p>
  <p>However, if we look at “TCEC Leela’s” evaluation at this point…</p>
  <p>…it gives a WDL breakdown of  11.3% White win, 88.7% Draw and 0% Black win (0.16 is the conversion of this expected score to centipawns which the TCEC does to be able to compare easily with Stockfish). That’s just a slight disadvantage. Why is that?</p>
  <p>Well, “TCEC Leela” moved 79…Bb5 very quickly, taking just 34 seconds for its move.</p>
  <p>In that short time, it searched 2.3M nodes (10M nodes less than my 24 mins’ effort) and this was presumably not enough for it to see all the dangers in the position!</p>
  </blockquote>
  <p>Position in question (I have taken the position one move earlier to have the board &quot;right side up&quot;. White played 79.Bd3):</p>
  <pre><code>[FEN &quot;5r1k/2nbq1rp/3p2p1/2pPp1P1/p3P3/P3NPQ1/1P2B2R/K6R w - - 3 79&quot;]
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-02-02T00:24:09.847Z
Answers.LastActivityDate: 2021-02-02T00:32:40.807Z
Answers.LastEditDate: 2021-02-02T00:32:40.807Z
Answers.UserDisplayName: Allure
Body: <p>I hope this question hasn't been asked before.</p>
  <p>I am looking for some examples where, say, the latest Stockfish or Komodo engine (or possibly some of the top NN engines) evaluates a position as plus or minus x for some time--possibly oscillating slightly near that point during that period--and then there is a big increase or drop in their evaluation.</p>
  <p>What might be a suitable &quot;for some time&quot; period? I want to say more than a minute, but not more than 1 hour. Also, what is a &quot;big&quot; change in evaluation? I am not so sure about that, but let's say something like 3.0.</p>
  <p>I am asking this question because, from my (very limited) experience, any dramatic changes in an engine's evaluation occur during the first 20 or so seconds. If an engine cannot find the deep tactics (or the deep positional play) during those first seconds then it is because the search tree has been pruned. (Am I right about this? I don't much about how chess engines operate.)</p>
  <p><strong>Edit:</strong>
  The drastic change in the evaluation should be a consequence of the introduction of a move not previously considered by the engine.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-11-26T00:25:24.410Z
Comments.Text: I remember a couple of analysis positions where that happened, let me see if I can find some. However on the note of pruning: pruning a line is not permanent, in fact if you let an engine calculate forever it will eventually search the entire tree without any pruning. (basically, the more you search, the less you prune early in the search tree; however you obviously gain new deeper parts of the tree that can be pruned)
Comments.UserDisplayName: koedem
Tags: <engines><evaluation>
Title: Big changes in engine's evaluation after considerable time
ThreadText: # Big changes in engine's evaluation after considerable time
  
  (Asked by: username_0 on 2020-11-25)
  
  I hope this question hasn't been asked before.
  
  I am looking for some examples where, say, the latest Stockfish or Komodo engine (or possibly some of the top NN engines) evaluates a position as plus or minus x for some time--possibly oscillating slightly near that point during that period--and then there is a big increase or drop in their evaluation.
  
  What might be a suitable "for some time" period? I want to say more than a minute, but not more than 1 hour. Also, what is a "big" change in evaluation? I am not so sure about that, but let's say something like 3.0.
  
  I am asking this question because, from my (very limited) experience, any dramatic changes in an engine's evaluation occur during the first 20 or so seconds. If an engine cannot find the deep tactics (or the deep positional play) during those first seconds then it is because the search tree has been pruned. (Am I right about this? I don't much about how chess engines operate.)
  
  **Edit:**
  The drastic change in the evaluation should be a consequence of the introduction of a move not previously considered by the engine.
  
  **Tags:** engines, evaluation
  
  > **Comments:**
  > **username_1**: I remember a couple of analysis positions where that happened, let me see if I can find some. However on the note of pruning: pruning a line is not permanent, in fact if you let an engine calculate forever it will eventually search the entire tree without any pruning. (basically, the more you search, the less you prune early in the search tree; however you obviously gain new deeper parts of the tree that can be pruned)  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_2 on 2020-11-27. Score: 8
  
  This happens often enough in engine tournaments that viewers have coined the terms "boom" and "moob" (for "eval explodes away from 0" and "eval implodes towards zero" respectively).
  
  Here's the [latest example of a boom](https://tcec-chess.com/#div=vsob19&game=38&season=19) from TCEC.
  
  ```
  [FEN "8/kp6/6B1/p3p3/3r3P/q5P1/2Q2PK1/8 w - - 0 59"]
  
  ```
  
  Leela, playing White, boomed from -0.46 the move before to -1.91 after playing 59. h5. That means this position (or one of the moves slightly prior) very likely meets your criteria, if you let Leela think for a sufficiently long time.
  
  You can also see from the principal variations of both engines that Leela did not anticipate Komodo's move on move 58 - Leela predicted 58...Qd5+, but Komodo played 58...Qxa3.
  
  If you're looking only for booms of more than +3, off the top of my head, [this](https://en.wikipedia.org/wiki/TCEC_Season_14#Game_85) was another memorable boom.
  
  ```
  [FEN "7Q/8/P3n1p1/2p3k1/6b1/5pb1/8/5K2 w - - 1 59"]
  
  ```
  
  Leela, playing White, evaluated the position after 59.a7 as -0.06, then boomed to -14.28 after 59...Bf5 (Leela predicted 59...Nc7).
  
  **Edited to add**: here's [an example of a Stockfish moob from -1.38 to 0.00](https://en.wikipedia.org/wiki/TCEC_Season_19#Game_9) after thinking for 4 minutes on TCEC hardware.
  
  ```
  [FEN "r4k1r/p1pb2bp/2pp1nq1/4P3/3P3Q/2N5/PPP3PP/R4RK1 w - - 3 16"]
  
  1.Rae1 Bf5
  
  ```
  
  Stockfish had the Black pieces and its eval dropped after ...Bf5. [Link to game](https://tcec-chess.com/#div=sf&game=9&season=19) with both engines' PVs if you want to see those.
  
  > **Comments:**
  > **username_0**: Yes, it's booms and moobs that I am interested in. (Thanks, by the way. I was not aware of these "terms".) How long did it take Leela to boom in the second one though? The latest Stockfish on my laptop finds mate in 22 in less than a minute. Do you have any examples where a boom (or moob) occurs after 2-3 minutes of evaluation?  
  > **username_2**: @username_0 the second one used a (by now) very old Leela net, which had very noticeable weaknesses. That's why Stockfish finds the mate quickly and the Leela missed it completely. The time taken for an engine to boom / moob can be quite a while because that time taken is completely dependent on the hardware you're running the program on. Nonetheless, I'm adding another example, this time of a Stockfish moob which took several minutes on TCEC hardware (i.e. on your hardware, Stockfish will probably take longer to moob).  
  
  ---
  
  ### Answer by username_1 on 2020-11-26. Score: 6
  
  This is a complicated position I had in a correspondence game. White to play and win. The engine evaluation rises and falls, occasionally hitting 0 again. Then it stays at roughly 1.5 for a long while until finally finding the breakthrough about 20 minutes in. Unsurprisingly, considering the full line is pretty insane.
  
  ```
  [FEN "r5r1/1np3bk/1p1p4/3P4/bpP2PPp/P3N2P/1q1BQ2K/4R1R1 w - - 0 33"]
  
  1. g5 Rge8 2. Qh5+ Kg8 3. Rg2 bxa3 4. Nf5 Rxe1 5. Bxe1 Bc2 6. Nh6+ Kh8 7. Ng4+ Kg8 8. Nf6+ Kf8 9. Nd7+ Ke7 10. Qg4 Nc5 11. g6 Rh8 12. Re2+ Kd8 13. Nb8 Ba4 14. Nc6+ Bxc6 15. dxc6 Qf6 16. Bf2 Re8 17. Ra2 Qe6 18. Bxh4+ Kc8 19. Re2 Kb8 20. Rxe6 Rxe6 21. Qg2 Re3 22. f5 Be5+ 23. Kg1 Rb3 24. Qa2 Bd4+ 25. Kf1 Rf3+ 26. Kg2 Rb3 27. f6 Rb2+ 28. Kf3 Rxa2 29. g7 Rg2 30. Kxg2 a2 31. g8=Q+ Ka7 32. f7 a1=Q 33. f8=Q Qg1+ 34. Kf3 Qd1+ 35. Kf4 Ne6+ 36. Qxe6 Qf1+ 37. Ke4 Qxf8 38. Qe7 Qxe7+ 39. Bxe7
  
  ```
  
  > **Comments:**
  > **username_3**: You may have to reset the move counter.. Start at 1. Idk if that's the only way but it seemed to work.  
  > **username_0**: I should have been more careful in the phrasing of the problem. The move g5 doesn't quite fit the bill of what I am looking for because it is one of the original moves suggested. The evaluation does change by more than 3.0 if you let the engine think quite a bit, but what I am really looking for is a drastic change in the evaluation suggested by a move not previously considered.  
  > **username_1**: @username_3 that worked indeed, thanks. A bit annoying but what can you do.  
  > **username_1**: @username_0 I see. There certainly are examples for that behavior too, however I don't know a fitting position off the top of my head. Let's see if I remember one later.  
  > **username_0**: Basically, I am looking for "aha" moments from the engine.  
  > **username_4**: If you fast-forward to the point where the line suggested by the original evaluation and the updated evaluation differ, would that give you the ah-ha moment? Presumably the engine spent a while on that first line before considering the second.  
  > **username_1**: @GS-ApologisetoMonica Good point, tried that now. Unfortunately while many positions on the way take a bit of time to switch the move, none of them take very long (so the accumulated time presumably simply contains the time of all switches along the way plus alternatives which also have to be calculated of course).  A better way to find such a position is probably one of those short puzzles that Stockfish doesn't find and then wait until, well, it finds it. But the time it takes for something like that might vary a lot / be rather long.  
  > **username_0**: @GS-ApologisetoMonica It would, yes. But how often is it that the line the engine has settled on within a minute differs drastically in evaluation from the line it thinks best within half an hour? Assuming the second line is different from the first, that is. I am guessing that this doesn't happen very often. Of particular interest would be the situation where a position is evaluated as nearly equal at the beginning, only to find a winning move (much) later on.  
  
  ---
  
  ### Answer by username_2 on 2021-02-02. Score: 1
  
  Adding another answer since it meets every requirement by the OP and I believe they'll be interested.
  
  ```
  [FEN "4nr2/5ppk/pn3q2/3ppNRP/1pr1P3/1P3P2/1KP3Q1/R7 w - - 0 42"]
  
  ```
  
  [Game 99 of the Season 20 TCEC Superfinal](https://tcec-chess.com/#div=sf&game=99&season=20)
  
  Leela had +0.03 eval with this principal variation:
  
  ```
  41....d5 42. bxc4 Nxc4+ 43. Kb1 dxe4 44. fxe4 Na3+ 45. Ka2 Qe6+ 46. Kb2 Nc4+ 47. Kb1 Rg8 48. Rg6 Qd7 49. Ka2 Qa4+ 50. Kb1 Qd7
  
  ```
  
  And it boomed to +10.59 after 42. Rg6 and 3 minutes 27s thought.
  
  Of course, it is TCEC hardware - their Leela ran at 92.1 knps. It's extremely unlikely OP (or anyone) can get anywhere near this speed. With weaker hardware, Leela's eval will be different.
  
  *However* it is worth emphasizing that OP's actual question (about the engine changing its mind during the search) is not so clear, because Leela could very well suffer from tunnel vision and completely miss 42. Rg6. That's not to say that Leela never changes its mind, it does, but it's not as simple as "run Leela for 3 minutes and it will boom". That's especially the case since what Leela sees in 3 minutes is heavily dependent on what hardware you run Leela on.
  
  Here's [an article](https://matthewsadler.me.uk/the-middlegame/anatomy-of-a-blunder/) by GM Matthew Sadler on a different blunder by Leela in the TCEC Season 20 superfinal illustrating the difficulties with the question, and possibly providing another interesting position. To quote:
  
  > Leave “my Leela” long enough (16.2M nodes searched) and its evaluation plummets: it sees that 79…Nb5 loses to the 80.Nf5 sacrifice and its assessment of its new top move (79…Bb5) is just 27.9% expected score (1 win, 3 draws and 6 losses). A big fall!
  >
  > However, if we look at “TCEC Leela’s” evaluation at this point…
  >
  > …it gives a WDL breakdown of 11.3% White win, 88.7% Draw and 0% Black win (0.16 is the conversion of this expected score to centipawns which the TCEC does to be able to compare easily with Stockfish). That’s just a slight disadvantage. Why is that?
  >
  > Well, “TCEC Leela” moved 79…Bb5 very quickly, taking just 34 seconds for its move.
  >
  > In that short time, it searched 2.3M nodes (10M nodes less than my 24 mins’ effort) and this was presumably not enough for it to see all the dangers in the position!
  
  Position in question (I have taken the position one move earlier to have the board "right side up". White played 79.Bd3):
  
  ```
  [FEN "5r1k/2nbq1rp/3p2p1/2pPp1P1/p3P3/P3NPQ1/1P2B2R/K6R w - - 3 79"]
  
  ```
  
  
  ---
  
  