Answers.Body: <p>Yes. <a href="https://groups.google.com/g/lczero/c/5tiNDWMEkes" rel="noreferrer">Antifish</a> was an attempt in 2018 to train a LC0 style network specifically to beat stockfish. It turns out that even when searching specifically for weaknesses, chess engines are really strong.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-27T03:19:50.697Z
Answers.LastActivityDate: 2023-05-27T03:19:50.697Z
Answers.UserDisplayName: Oscar Smith
Answers.Body: <p>The problem with this question is that it's not clear how an adversarial engine would work. In a comment you write:</p>
  <blockquote>
  <p>making moves where the payoff is beyond the chess engine's calculable horizon</p>
  </blockquote>
  <p>But what is &quot;calculable horizon&quot; to you? Let's say you define it as &quot;the engine does not find the refutation after thinking for 10 minutes on a $1000 2023 desktop&quot;. Now what if:</p>
  <ol>
  <li>Someone lets the engine think for 20 minutes</li>
  <li>Someone uses a $2000 desktop</li>
  </ol>
  <p>And the engine finds the refutation - is the adversarial engine still adversarial? If yes, then the definition of &quot;adversarial&quot; is very arbitrary. If no, then it's not well-defined in the first place.</p>
  <p>Accordingly, the question is too vague to answer.</p>
  <p>PS: There are positions where the engine does not find the key move after thinking for a &quot;very long time&quot; (inverted commas because this term is also not well-defined, c.f. the above), but they're rare, so an engine that makes a good move in rare positions at the cost of making not-so-good moves in more common positions will lose elo in self-play.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-24T15:01:34.970Z
Answers.Text: Horizon is a parameter for the adversarial engine.  It is adversarial against a specific version of a specific chess engine, such as stockfish.  Someone did this sort of thing recently for Go.  I've also seen specific instances for chess where chess engines fail, due to the horizon problem.
Answers.UserDisplayName: yters
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-25T02:12:54.533Z
Answers.Text: I'm not familiar with computer Go, but I'm pretty familiar with computer Chess, and I don't understand how horizon can be a parameter for the adversarial engine. Can you explain?
Answers.UserDisplayName: Allure
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-25T13:28:21.150Z
Answers.Text: Tell the adversarial engine the target has enough computational resources to calculate N-ply deep. Or, find situations where pruning is not effective, so N is always small regardless of the chess engine.
Answers.UserDisplayName: yters
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-25T14:10:27.110Z
Answers.Text: @yters but N depends on position. And pruning is extremely effective in chess engines, they're virtually unbeatable from the starting position. The positions they misevaluate are few and far between. Maybe people have tried to create adversaries for chess engines (and some of them are linked in a comment), but there is no known way to reliably beat SF/Lc0, not from the starting position.
Answers.UserDisplayName: Allure
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-26T19:01:48.783Z
Answers.Text: this answer of yours is the sort of thing I'm asking about.  https://chess.stackexchange.com/questions/33098/big-changes-in-engines-evaluation-after-considerable-time/33952#33952 Have people researched ways to find booms for a specific chess engine?  Then they could use the boom to beat chess engines that don't calculate far enough.
Answers.UserDisplayName: yters
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-26T23:43:09.637Z
Answers.Text: @yters nobody is researching ways to find "booms" for a specific chess engine, then, and I'd be highly skeptical anyone could use such ways to beat chess engines that don't calculate far enough, for reasons described in this answer.
Answers.UserDisplayName: Allure
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-24T05:50:44.443Z
Answers.LastActivityDate: 2023-05-24T05:50:44.443Z
Answers.UserDisplayName: Allure
Body: <p>Even with all the compute in the world, chess engines cannot compute very deeply.  So, they have to make use of pruning heuristics and discard moves from the analysis.</p>
  <p>Seems possible that even the best heuristics will mistakenly discard good moves, and conversely go down bad paths that initially look good from a heuristic standpoint.</p>
  <p>Have researchers tried to identify when this tends to happen, and if this can be taken advantage of, similar to adversarial examples in Deep Learning?  E.g. an expert with deep understanding of a chess engine internals could concoct a series of moves to force the chess engine to make bad moves.</p>
  <p>I play chess.com occasionally, and I've noticed something like this happening, where the chess engine will evaluate the same move as exceedingly good, and then as exceedingly bad.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-11-02T21:37:35.587Z
Comments.Text: There are certain engines out there which are supposedly designed to play against Stockfish in particular. Though I haven't heard much about them in the past couple years. It's probably harder now that Stockfish uses a neural net for its evaluation, as opposed to hand-crafted heuristics.
Comments.UserDisplayName: Inertial Ignorance
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-11-02T21:42:20.753Z
Comments.Text: I did notice the use of neural networks, which also led to the question about adversarial examples.  The imagery deep learning networks are easily fooled by minor image tweaks.  I was wondering if the chess engine neural networks are similarly brittle.
Comments.UserDisplayName: yters
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-11-03T11:33:39.950Z
Comments.Text: @yters In general, neural nets that are trained though self-play reinforcement are much more robust against adversarial inputs than supervised models constrained by the size and quality of the annotated datasets on which their learning was based. That said, potential weaknesses can still remain due to the relation between feature encoding and the horizon effect on learning rate. For example, early Alphago-inspired go engines would tend to misjudge [ladders](https://en.wikipedia.org/wiki/Ladder_(Go)), whereas recent stronger engines avoid the problem by encoding the presence/absence of...
Comments.UserDisplayName: Will
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-11-03T11:35:36.780Z
Comments.Text: ...ladders as additional input features (providing the model with new "vocabulary" so to speak). So if I were to look at finding such "exploits", I'd look at (lack of) feature encoding first.
Comments.UserDisplayName: Will
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-11-09T02:14:02.123Z
Comments.Text: How would this look like in practice? Would you say an agent that memorizes the games Stockfish loses and then repeats them ad infinitum (only works since chess engines are semi-deterministic) is an adversarial agent?
Comments.UserDisplayName: Allure
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-11-09T15:30:32.313Z
Comments.Text: Yes, but I am interested in something more like adversarial strategies.  I.e. making moves where the payoff is beyond the chess engine's calculable horizon.
Comments.UserDisplayName: yters
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-04-20T05:16:46.470Z
Comments.Text: https://chess.stackexchange.com/questions/41867/a-possibility-to-beat-a-chess-computer This was actually asked and answered recently, you might be interested.
Comments.UserDisplayName: Allure
Tags: <engines>
Title: Has any researcher tried adversarial examples against chess engines?
ThreadText: # Has any researcher tried adversarial examples against chess engines?
  
  (Asked by: username_0 on 2022-11-02)
  
  Even with all the compute in the world, chess engines cannot compute very deeply. So, they have to make use of pruning heuristics and discard moves from the analysis.
  
  Seems possible that even the best heuristics username_2 mistakenly discard good moves, and conversely go down bad paths that initially look good from a heuristic standpoint.
  
  Have researchers tried to identify when this tends to happen, and if this can be taken advantage of, similar to adversarial examples in Deep Learning? E.g. an expert with deep understanding of a chess engine internals could concoct a series of moves to force the chess engine to make bad moves.
  
  I play chess.com occasionally, and I've noticed something like this happening, where the chess engine username_2 evaluate the same move as exceedingly good, and then as exceedingly bad.
  
  **Tags:** engines
  
  > **Comments:**
  > **username_1**: There are certain engines out there which are supposedly designed to play against Stockfish in particular. Though I haven't heard much about them in the past couple years. It's probably harder now that Stockfish uses a neural net for its evaluation, as opposed to hand-crafted heuristics.  
  > **username_0**: I did notice the use of neural networks, which also led to the question about adversarial examples.  The imagery deep learning networks are easily fooled by minor image tweaks.  I was wondering if the chess engine neural networks are similarly brittle.  
  > **username_2**: @username_0 In general, neural nets that are trained though self-play reinforcement are much more robust against adversarial inputs than supervised models constrained by the size and quality of the annotated datasets on which their learning was based. That said, potential weaknesses can still remain due to the relation between feature encoding and the horizon effect on learning rate. For example, early Alphago-inspired go engines would tend to misjudge [ladders](https://en.wikipedia.org/wiki/Ladder_(Go)), whereas recent stronger engines avoid the problem by encoding the presence/absence of...  
  > **username_2**: ...ladders as additional input features (providing the model with new "vocabulary" so to speak). So if I were to look at finding such "exploits", I'd look at (lack of) feature encoding first.  
  > **username_3**: How would this look like in practice? Would you say an agent that memorizes the games Stockfish loses and then repeats them ad infinitum (only works since chess engines are semi-deterministic) is an adversarial agent?  
  > **username_0**: Yes, but I am interested in something more like adversarial strategies.  I.e. making moves where the payoff is beyond the chess engine's calculable horizon.  
  > **username_3**: https://chess.stackexchange.com/questions/41867/a-possibility-to-beat-a-chess-computer This was actually asked and answered recently, you might be interested.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_4 on 2023-05-27. Score: 5
  
  Yes. [Antifish](https://groups.google.com/g/lczero/c/5tiNDWMEkes) was an attempt in 2018 to train a LC0 style network specifically to beat stockfish. It turns out that even when searching specifically for weaknesses, chess engines are really strong.
  
  
  ---
  
  ### Answer by username_3 on 2023-05-24. Score: 1
  
  The problem with this question is that it's not clear how an adversarial engine would work. In a comment you write:
  
  > making moves where the payoff is beyond the chess engine's calculable horizon
  
  But what is "calculable horizon" to you? Let's say you define it as "the engine does not find the refutation after thinking for 10 minutes on a $1000 2023 desktop". Now what if:
  
  1. Someone lets the engine think for 20 minutes
  2. Someone uses a $2000 desktop
  
  And the engine finds the refutation - is the adversarial engine still adversarial? If yes, then the definition of "adversarial" is very arbitrary. If no, then it's not well-defined in the first place.
  
  Accordingly, the question is too vague to answer.
  
  PS: There are positions where the engine does not find the key move after thinking for a "very long time" (inverted commas because this term is also not well-defined, c.f. the above), but they're rare, so an engine that makes a good move in rare positions at the cost of making not-so-good moves in more common positions username_2 lose elo in self-play.
  
  > **Comments:**
  > **username_0**: Horizon is a parameter for the adversarial engine.  It is adversarial against a specific version of a specific chess engine, such as stockfish.  Someone did this sort of thing recently for Go.  I've also seen specific instances for chess where chess engines fail, due to the horizon problem.  
  > **username_3**: I'm not familiar with computer Go, but I'm pretty familiar with computer Chess, and I don't understand how horizon can be a parameter for the adversarial engine. Can you explain?  
  > **username_0**: Tell the adversarial engine the target has enough computational resources to calculate N-ply deep. Or, find situations where pruning is not effective, so N is always small regardless of the chess engine.  
  > **username_3**: @username_0 but N depends on position. And pruning is extremely effective in chess engines, they're virtually unbeatable from the starting position. The positions they misevaluate are few and far between. Maybe people have tried to create adversaries for chess engines (and some of them are linked in a comment), but there is no known way to reliably beat SF/Lc0, not from the starting position.  
  > **username_0**: this answer of yours is the sort of thing I'm asking about.  https://chess.stackexchange.com/questions/33098/big-changes-in-engines-evaluation-after-considerable-time/33952#33952 Have people researched ways to find booms for a specific chess engine?  Then they could use the boom to beat chess engines that don't calculate far enough.  
  > **username_3**: @username_0 nobody is researching ways to find "booms" for a specific chess engine, then, and I'd be highly skeptical anyone could use such ways to beat chess engines that don't calculate far enough, for reasons described in this answer.  
  
  ---
  
  