Answers.Body: <p>Interesting question. I think it depends on how much bottom-up intelligence the engine has. For example, AlphaZero was given no explicit heuristics, but was able to infer plenty of strategy by playing itself millions of times and learning that way.</p>
  <p>An engine with explicit heuristics can also exhibit additional strategies you didn't program into it. As a non-chess example, I've programmed a Connect Four engine with a number of heuristics. Using these and its search algorithm, it then tended to favour clumping pieces together in the centre when given the opportunity. While you might argue the engine doesn't necessarily &quot;understand&quot; this bottom up behaviour it's doing here (does a program really understand anything it does?), it's still playing so that for all intents and purposes, it values clumping pieces in the centre.</p>
  <p>In my opinion, if a chess engine is able to infer some heuristic A (or behave <em>as if</em> it understands this heuristic, even if it's just bottom-up behaviour), given a set of explicit heuristics X, then the question comes down to whether you can describe A better than the engine would understand it on its own. If you're making your own chess engine then I think it's best to make some explicit heuristic for opposite-side castling. For the top engines though I'm not sure whether or not they do this, for the reasons discussed. It might be hard to understand the raised importance of king safety without calculating far ahead and seeing pawn storms happen, so it's reasonable to think these engines would have something given to them about opposite-side castling.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-12T02:15:17.797Z
Answers.Text: Your connect four engine doesn't sound like it inferred any heuristics. It sounds like it just frequently computed the best move to be near the center, from which *you* then inferred the idea of a heuristic that your program did not actually use.
Answers.UserDisplayName: user2357112
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-12T06:29:22.500Z
Answers.Text: @user2357112supportsMonica Well a computer program has to obey its code, and do nothing else. So it can't write additional code for some extra heuristic, but it can behave as if it has this heuristic. At that point you can debate whether the engine "understands" this extra behavior it's exhibiting, and it comes down to what understanding even means. I edited the second paragraph for more clarity.
Answers.UserDisplayName: Inertial Ignorance
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-13T02:37:45.580Z
Answers.Text: AFAIK your connect four engine does not employ any heuristics. Stockfish or similar engines have hard-coded strategies in the program, they are not just "behave as if it has this heuristic". On the other hand, your program did not move to the center intentionally. It's just you who concluded its moves.
Answers.UserDisplayName: TwilightSparkle
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-13T03:57:23.257Z
Answers.Text: @HighlyRadioactive I have hard coded strategies/heuristics for certain aspects of Connect Four, but not particularly for clumping pieces in the centre. This was just for an analogy to a hypothetical chess engine that doesn't have an explicit heuristic for opposite side castling (instead it would have many other heuristics), but behaves as if it does.
Answers.UserDisplayName: Inertial Ignorance
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-13T04:05:34.100Z
Answers.Text: Clumping pieces in the centre is not a heuristics, just a *behaviour*. The OP is asking "should opposite castling be a heuristics".
Answers.UserDisplayName: TwilightSparkle
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-13T04:15:22.953Z
Answers.Text: @HighlyRadioactive The entire point I'm making is that if you want something to be a behaviour (such as opposite-side castling), it doesn't necessarily need to be a heuristic.
Answers.UserDisplayName: Inertial Ignorance
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-13T19:26:21.850Z
Answers.Text: @InertialIgnorance  _"In my opinion, if a chess engine is able to infer some heuristic A, given a set of explicit heuristics X"_ I'm sorry, but emergent behavior is simply not the same as 'inferring heuristics'. What you describe (inferring heuristics) is something significantly more advanced that would be akin to a logic engine or general artificial intelligence, or a part of an engine that does explicit data mining on previous games in preparation for future games. That's not what's being described here though.
Answers.UserDisplayName: orlp
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-13T20:20:33.873Z
Answers.Text: @orlp Good point, I edited that sentence.
Answers.UserDisplayName: Inertial Ignorance
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-13T23:29:00.533Z
Answers.Text: By "bottom-up" do you mean "emergent"?
Answers.UserDisplayName: OrangeDog
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-14T00:42:52.110Z
Answers.Text: @OrangeDog Yes.
Answers.UserDisplayName: Inertial Ignorance
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-11T03:05:14.487Z
Answers.LastActivityDate: 2020-08-13T20:20:14.963Z
Answers.LastEditDate: 2020-08-13T20:20:14.963Z
Answers.UserDisplayName: Inertial Ignorance
Answers.Body: <p>I'd like to make explicit a point that's been hinted at by the existing answers.</p>
  <p>I'm from an AI, not specifically a chess, background. The usual approach to problems in games like this is to treat the game as a <a href="https://en.wikipedia.org/wiki/Markov_property" rel="nofollow noreferrer">Markov process</a>: a system in which future states are (maybe probabilistically) determined entirely by the current state, with no need for an explicit &quot;memory&quot; of how one arrived at the current state. Everything there is to know is represented in the observed board state.</p>
  <p>In this context, a chess engine wouldn't need to be aware of any history at all. Maybe the players just randomly placed pieces on the board until they arrived at the current board state: the engine doesn't know or care if it was switched on mid-game or whatever, it doesn't rely on remembering anyone's past moves. This makes sense: suppose your engine <em>did</em> pay extra attention to the fact that opposite-side castling occurred. How long should that matter for? Should we learn separate evaluations for &quot;the pieces are here, and OSC occurred [1, 2, 5...] turns ago&quot;?</p>
  <p>Humans playing human opponents may develop a model of opponent psychology: maybe you can tell your opponent's attention is fixated on a certain part of the board, or predict what they're planning based on their prior moves, their tempo, the decisiveness with which they handle the pieces. For a computer to consider this would require adding far too much extra uncertainty and supposition into a problem that's already really really hard just from the board state combinatorics--assuming you could even find a way to communicate those factors to the computer. And learning from it would mean learning <em>bad data about board position</em>--because the computer would have to separate the learned model of its (individual) opponent's psychology from the learned evaluation of how good a particular board position is.</p>
  <p>Instead the computer plays like one of those chess masters who plays 15 different opponents at once: by ignoring the history of the game and just making moves based on where things are now.</p>
  <p>My guess is that the chess experts you're alluding to are giving a heuristic for evaluating board states. Now, it's possible to imagine an engine which looks at board states and classifies them according to various properties: &quot;OSC has occurred,&quot; &quot;[I/my opponent] control[s] the center of the board,&quot; etc., and then predicts moves based on those properties as <a href="https://en.wikipedia.org/wiki/Feature_(machine_learning)" rel="nofollow noreferrer">features</a> (to reduce the overwhelming combinatorial complexity of board states &amp; capture some of their natural symmetries). You now have two learning problems: first, how to identify a board state that meets a given heuristic; and second, how to use that set of heuristics to determine how to play. This might have been interesting twenty years ago, but it looks like the state of the art is past that (although this is probably happening in some sense in <a href="https://en.wikipedia.org/wiki/Artificial_neural_network" rel="nofollow noreferrer">neural network</a>-based chess engines; but as is typical for neural networks, we humans don't recognize anything coherent in the features that are learned in the middle layers of the network.)</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-14T09:05:42.263Z
Answers.Text: TL;DR for all the answers: Markov property.
Answers.UserDisplayName: Mateen Ulhaq
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-14T13:15:40.990Z
Answers.Text: @MateenUlhaq yes. I was hoping to also give a bit of an explanation for what that is & the rationale involved as well, though.
Answers.UserDisplayName: Tiercelet
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-13T15:13:03.950Z
Answers.LastActivityDate: 2020-08-14T13:16:47.317Z
Answers.LastEditDate: 2020-08-14T13:16:47.317Z
Answers.UserDisplayName: Tiercelet
Answers.Body: <blockquote>
  <p>Is opposite side castling significant enough that a chess engine should think about such positions differently from how it would normally?</p>
  </blockquote>
  <p>I'm not sure your question makes sense. Obviously, a chess engine should take into account the game state. Opposite side castling is part of the game state. So obviously it should influence the evaluation. But what does it mean to &quot;think about such positions differently from how it would normally?&quot; Like, the engine does a &quot;general&quot; evaluation that doesn't take into account opposite side castling, and then &quot;adds in&quot; a factor that takes into account opposite side castling? What would that mean? That there's been opposite side castling is part of the position. What would it mean to to evaluate the position except for the fact that there's been opposite side castling? I guess if your basic analysis just looks at material strength, and doesn't look at position, then adding in opposite side castling could be useful, but any engine advanced enough to be at all competitive is already going to be deeply positional.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-12T19:53:40.783Z
Answers.Text: I gave an example of lowering the weight of material and increasing the weight of king safety as a change that could be made when opposite side castling occurs. Similar to how engines use different evaluation for endgames and middlegames, I thought it could be useful to have different evaluation for opposite side castled positions since I've heard it changes the dynamic of the game. As I've learned through this post though, that is unnecessary as other heuristics can account for it. As user Allure pointed out, menace and pawn shield.
Answers.UserDisplayName: Tauist
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-12T06:18:56.120Z
Answers.LastActivityDate: 2020-08-12T06:18:56.120Z
Answers.UserDisplayName: Acccumulation
Answers.Body: <p>I'm pretty sure Stockfish doesn't have explicit code that handles opposite-side castling. What it does have is:</p>
  <ul>
  <li>Some kind of &quot;menace&quot; score for enemy pawns advancing against our king. The closer they get, the more dangerous Stockfish thinks they are.</li>
  <li>Some kind of &quot;pawn shield&quot; score for friendly pawns in front of our king. The fewer there are, the more dangerous Stockfish thinks the position is.</li>
  </ul>
  <p>Therefore you get kind of a balancing act: if we castle on the same side, then we could advance our pawns against their king (getting the &quot;menace bonus&quot;), but it would reduce our pawn shield (and therefore losing the &quot;pawn shield bonus&quot;). The converse does not hold in opposite side castling, which therefore encourages the engine to push pawns against the enemy king.</p>
  <p>Of course, the details are more complicated than this, but that's the gist.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-13T13:09:46.363Z
Answers.Text: "The closer they get, the more dangerous Stockfish thinks they are." That makes sense, doesn't it? A pawn at it's starting position is usually a lot less dangerous than a pawn at the center of the board.
Answers.UserDisplayName: Mast
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-13T13:24:06.740Z
Answers.Text: @Mast yeah, it does. Things like that are why we say Stockfish incorporates human knowledge.
Answers.UserDisplayName: Allure
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-11T04:12:28.540Z
Answers.LastActivityDate: 2020-08-11T04:12:28.540Z
Answers.UserDisplayName: Allure
Body: <p>I often hear chess experts and chess articles say that when opposite side castling occurs on the board that it is important to launch an attack on the enemy player as fast as possible. Up to the point where giving up a couple pawns or a piece could be justified.</p>
  <p>In other words, it seems like castling opposite sides changes the dynamic of the game. How much this change is I'm not sure of. So here is my question:</p>
  <blockquote>
  <p>Is opposite side castling significant enough that a chess engine should think about such positions differently from how it would normally?</p>
  </blockquote>
  <p>For example, a chess engine could lower the importance of material and up the importance of king safety in its evaluation when opposite castling occurs.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-08-11T09:32:05.283Z
Comments.Text: It's certainly not conclusive, but I did a search for the terms "castle", "castl", "opposite" and "opp" in the Stockfish code and found nothing related opposite castling.
Comments.UserDisplayName: emdio
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-08-12T13:50:41.810Z
Comments.Text: The philosophy of neural-network based approaches, such as Leela, are that the engine should not be given any preconceived notions at all, but would presumably learn these kinds of heuristics through experience.
Comments.UserDisplayName: usul
Tags: <engines><castling><evaluation>
Title: Should a chess engine be aware of opposite-side castling?
ThreadText: # Should a chess engine be aware of opposite-side castling?
  
  (Asked by: username_0 on 2020-08-11)
  
  I often hear chess experts and chess articles say that when opposite side castling occurs on the board that it is important to launch an attack on the enemy player as fast as possible. Up to the point where giving up a couple pawns or a piece could be justified.
  
  In other words, it seems like castling opposite sides changes the dynamic of the game. How much this change is I'm not sure of. So here is my question:
  
  > Is opposite side castling significant enough that a chess engine should think about such positions differently from how it would normally?
  
  For example, a chess engine could lower the importance of material and up the importance of king safety in its evaluation when opposite castling occurs.
  
  **Tags:** engines, castling, evaluation
  
  > **Comments:**
  > **username_1**: It's certainly not conclusive, but I did a search for the terms "castle", "castl", "opposite" and "opp" in the Stockfish code and found nothing related opposite castling.  
  > **username_2**: The philosophy of neural-network based approaches, such as Leela, are that the engine should not be given any preconceived notions at all, but would presumably learn these kinds of heuristics through experience.  
  
  ---
  
  ## 4 answers
  
  ### Answer by username_11 on 2020-08-11. Score: 38 | Accepted answer
  
  I'm pretty sure Stockfish doesn't have explicit code that handles opposite-side castling. What it does have is:
  
  * Some kind of "menace" score for enemy pawns advancing against our king. The closer they get, the more dangerous Stockfish thinks they are.
  * Some kind of "pawn shield" score for friendly pawns in front of our king. The fewer there are, the more dangerous Stockfish thinks the position is.
  
  Therefore you get kind of a balancing act: if we castle on the same side, then we could advance our pawns against their king (getting the "menace bonus"), but it would reduce our pawn shield (and therefore losing the "pawn shield bonus"). The converse does not hold in opposite side castling, which therefore encourages the engine to push pawns against the enemy king.
  
  Of course, the details are more complicated than this, but that's the gist.
  
  > **Comments:**
  > **username_12**: "The closer they get, the more dangerous Stockfish thinks they are." That makes sense, doesn't it? A pawn at it's starting position is usually a lot less dangerous than a pawn at the center of the board.  
  > **username_11**: @username_12 yeah, it does. Things like that are why we say Stockfish incorporates human knowledge.  
  
  ---
  
  ### Answer by username_3 on 2020-08-11. Score: 16
  
  Interesting question. I think it depends on how much bottom-up intelligence the engine has. For example, AlphaZero was given no explicit heuristics, but was able to infer plenty of strategy by playing itself millions of times and learning that way.
  
  An engine with explicit heuristics can also exhibit additional strategies you didn't program into it. As a non-chess example, I've programmed a Connect Four engine with a number of heuristics. Using these and its search algorithm, it then tended to favour clumping pieces together in the centre when given the opportunity. While you might argue the engine doesn't necessarily "understand" this bottom up behaviour it's doing here (does a program really understand anything it does?), it's still playing so that for all intents and purposes, it values clumping pieces in the centre.
  
  In my opinion, if a chess engine is able to infer some heuristic A (or behave *as if* it understands this heuristic, even if it's just bottom-up behaviour), given a set of explicit heuristics X, then the question comes down to whether you can describe A better than the engine would understand it on its own. If you're making your own chess engine then I think it's best to make some explicit heuristic for opposite-side castling. For the top engines though I'm not sure whether or not they do this, for the reasons discussed. It might be hard to understand the raised importance of king safety without calculating far ahead and seeing pawn storms happen, so it's reasonable to think these engines would have something given to them about opposite-side castling.
  
  > **Comments:**
  > **username_4**: Your connect four engine doesn't sound like it inferred any heuristics. It sounds like it just frequently computed the best move to be near the center, from which *you* then inferred the idea of a heuristic that your program did not actually use.  
  > **username_3**: @username_4supportsMonica Well a computer program has to obey its code, and do nothing else. So it can't write additional code for some extra heuristic, but it can behave as if it has this heuristic. At that point you can debate whether the engine "understands" this extra behavior it's exhibiting, and it comes down to what understanding even means. I edited the second paragraph for more clarity.  
  > **username_5**: AFAIK your connect four engine does not employ any heuristics. Stockfish or similar engines have hard-coded strategies in the program, they are not just "behave as if it has this heuristic". On the other hand, your program did not move to the center intentionally. It's just you who concluded its moves.  
  > **username_3**: @HighlyRadioactive I have hard coded strategies/heuristics for certain aspects of Connect Four, but not particularly for clumping pieces in the centre. This was just for an analogy to a hypothetical chess engine that doesn't have an explicit heuristic for opposite side castling (instead it would have many other heuristics), but behaves as if it does.  
  > **username_5**: Clumping pieces in the centre is not a heuristics, just a *behaviour*. The OP is asking "should opposite castling be a heuristics".  
  > **username_3**: @HighlyRadioactive The entire point I'm making is that if you want something to be a behaviour (such as opposite-side castling), it doesn't necessarily need to be a heuristic.  
  > **username_6**: @InertialIgnorance  _"In my opinion, if a chess engine is able to infer some heuristic A, given a set of explicit heuristics X"_ I'm sorry, but emergent behavior is simply not the same as 'inferring heuristics'. What you describe (inferring heuristics) is something significantly more advanced that would be akin to a logic engine or general artificial intelligence, or a part of an engine that does explicit data mining on previous games in preparation for future games. That's not what's being described here though.  
  > **username_3**: @username_6 Good point, I edited that sentence.  
  > **username_7**: By "bottom-up" do you mean "emergent"?  
  > **username_3**: @username_7 Yes.  
  
  ---
  
  ### Answer by username_8 on 2020-08-13. Score: 7
  
  I'd like to make explicit a point that's been hinted at by the existing answers.
  
  I'm from an AI, not specifically a chess, background. The usual approach to problems in games like this is to treat the game as a [Markov process](https://en.wikipedia.org/wiki/Markov_property): a system in which future states are (maybe probabilistically) determined entirely by the current state, with no need for an explicit "memory" of how one arrived at the current state. Everything there is to know is represented in the observed board state.
  
  In this context, a chess engine wouldn't need to be aware of any history at all. Maybe the players just randomly placed pieces on the board until they arrived at the current board state: the engine doesn't know or care if it was switched on mid-game or whatever, it doesn't rely on remembering anyone's past moves. This makes sense: suppose your engine *did* pay extra attention to the fact that opposite-side castling occurred. How long should that matter for? Should we learn separate evaluations for "the pieces are here, and OSC occurred [1, 2, 5...] turns ago"?
  
  Humans playing human opponents may develop a model of opponent psychology: maybe you can tell your opponent's attention is fixated on a certain part of the board, or predict what they're planning based on their prior moves, their tempo, the decisiveness with which they handle the pieces. For a computer to consider this would require adding far too much extra uncertainty and supposition into a problem that's already really really hard just from the board state combinatorics--assuming you could even find a way to communicate those factors to the computer. And learning from it would mean learning *bad data about board position*--because the computer would have to separate the learned model of its (individual) opponent's psychology from the learned evaluation of how good a particular board position is.
  
  Instead the computer plays like one of those chess masters who plays 15 different opponents at once: by ignoring the history of the game and just making moves based on where things are now.
  
  My guess is that the chess experts you're alluding to are giving a heuristic for evaluating board states. Now, it's possible to imagine an engine which looks at board states and classifies them according to various properties: "OSC has occurred," "[I/my opponent] control[s] the center of the board," etc., and then predicts moves based on those properties as [features](https://en.wikipedia.org/wiki/Feature_(machine_learning)) (to reduce the overwhelming combinatorial complexity of board states & capture some of their natural symmetries). You now have two learning problems: first, how to identify a board state that meets a given heuristic; and second, how to use that set of heuristics to determine how to play. This might have been interesting twenty years ago, but it looks like the state of the art is past that (although this is probably happening in some sense in [neural network](https://en.wikipedia.org/wiki/Artificial_neural_network)-based chess engines; but as is typical for neural networks, we humans don't recognize anything coherent in the features that are learned in the middle layers of the network.)
  
  > **Comments:**
  > **username_9**: TL;DR for all the answers: Markov property.  
  > **username_8**: @MateenUlhaq yes. I was hoping to also give a bit of an explanation for what that is & the rationale involved as well, though.  
  
  ---
  
  ### Answer by username_10 on 2020-08-12. Score: 5
  
  > Is opposite side castling significant enough that a chess engine should think about such positions differently from how it would normally?
  
  I'm not sure your question makes sense. Obviously, a chess engine should take into account the game state. Opposite side castling is part of the game state. So obviously it should influence the evaluation. But what does it mean to "think about such positions differently from how it would normally?" Like, the engine does a "general" evaluation that doesn't take into account opposite side castling, and then "adds in" a factor that takes into account opposite side castling? What would that mean? That there's been opposite side castling is part of the position. What would it mean to to evaluate the position except for the fact that there's been opposite side castling? I guess if your basic analysis just looks at material strength, and doesn't look at position, then adding in opposite side castling could be useful, but any engine advanced enough to be at all competitive is already going to be deeply positional.
  
  > **Comments:**
  > **username_0**: I gave an example of lowering the weight of material and increasing the weight of king safety as a change that could be made when opposite side castling occurs. Similar to how engines use different evaluation for endgames and middlegames, I thought it could be useful to have different evaluation for opposite side castled positions since I've heard it changes the dynamic of the game. As I've learned through this post though, that is unnecessary as other heuristics can account for it. As user username_11 pointed out, menace and pawn shield.  
  
  ---
  
  