Answers.Body: <p>Up to date reference is practically impossible to give, since engines and hardware change on yearly basis.</p>
  <p>You identified most important conditions, but I would like to add two more:</p>
  <ol>
  <li>Presence of tablebases (vastly improves evaluation depth in late endgames);</li>
  <li><a href="https://www.chessprogramming.org/NNUE" rel="nofollow noreferrer">NNUE</a> (slower) or hand-tuned evaluation function (faster).</li>
  </ol>
  <p>Introduction of NNUE techniques vastly improved accuracy of evaluation function, but also made it slower. If you compare stockfish versions before NNUE and after you will see in same time frame difference of as many as 10-20 ply. Newest stockfish is still stronger than older, because it gains more in evaluation accuracy than loses in depth (in most positions! in some endgames this tradeoff is not so good).</p>
  <p>From my experience (running time 1-2 minutes):</p>
  <ul>
  <li>SF before NNUE in middlegame on reasonably strong computer: 30-40 ply</li>
  <li>SF with NNUE in middlegame on reasonably strong computer: 20 ply</li>
  <li>SF with NNUE in middlegame with 64 cores and 128gb ram: 30-40 ply</li>
  <li>SF with 6 piece tablebases in endgame with ~8 pieces: 60+ ply or (more often) forced win/draw found.</li>
  </ul>
  <p>Note that in early openings engines are very unreliable, no matter the (contemporarily achievable) depth.</p>
  <hr />
  <p>As <a href="https://chess.stackexchange.com/users/27564/polytropos">@Polytropos</a> pointed out in comment, meaning of depth depends on choice of tree search algorithm. Possible moves and possible replies to them (and replies to them ...) can be arrayed into tree structure, so question of finding the best move is question of finding route to victory. Since this tree is too big to cover entirely we need to cheat - stop searching before we find the goal, evaluate positions and calculate approximate scores.</p>
  <p>Additionaly chess is a zero sum game - this means, that your position can only be as good for you as it is bad for your opponent (and vice versa). This means that <a href="https://www.chessprogramming.org/Alpha-Beta" rel="nofollow noreferrer">minimax</a> search can be used.</p>
  <p>Detailed explanation of tree search algorithms is out of scope for this answer. If interested, consult pages for <a href="https://www.chessprogramming.org/Alpha-Beta" rel="nofollow noreferrer">minimax</a>, <a href="https://www.chessprogramming.org/Alpha-Beta" rel="nofollow noreferrer">Alpha-Beta pruning</a> and <a href="https://www.chessprogramming.org/Monte-Carlo_Tree_Search" rel="nofollow noreferrer">Monte-Carlo tree search (MCTS)</a>. I will focus on practical consequences of choice of algorithm. Theoretically given infinite time they all lead to optimal solution.</p>
  <p>Minimax in pure form cannot be used practically, since it's too time intensive. So in practice it's optimized by Alpha-Beta pruning. This way of searching is used by <a href="https://stockfishchess.org/" rel="nofollow noreferrer">Stockfish</a>, currently strongest publically available engine. <strong>Depth here means number of plys before reaching evaluation, with some moves ignored at each ply based on preliminary evaluations.</strong> Practically it optimizes for greater depth by sacrificing some breadth.</p>
  <p>Consequence is that it may miss some good moves, which look bad to preliminary evaluation. Consequently human skill still matters in engine-assisted correspondence chess, since humans can force engine to consider moves that it would disregard to early (most commonly sacrifices with long-term positional compensation). It also leads to engines being weaker in (at least quiet) openings compared to (at least forcing) middlegames, since breadth is larger in openings, so more of it has to be pruned.</p>
  <p>MCTS is used most notably by AlphaZero and <a href="https://lczero.org/" rel="nofollow noreferrer">Lc0</a>. It's based on probabilistic methods. Practically it leads to lower search depths but better breadth than minimax with Alpha-Beta pruning.</p>
  <p>Consequently play of these engines is slightly better in openings, but slightly worse in endgames (but this may also be attributed to evaluation function).</p>
  <p><strong>Depth for Lc0 is average depth reached for move,</strong> based on considered sample.</p>
  <p><strong>In short, don't compare depths reached by one engine to depths reached by different engine. Depth is inaccurate heuristic for accuracy, as such it can only tell you, that if same engine reaches depth 20 and depth 40, evaluation at depth 40 is most probably more accurate.</strong></p>
  <hr />
  <p>More pedantically Lc0 uses <a href="https://lczero.org/dev/wiki/technical-explanation-of-leela-chess-zero/" rel="nofollow noreferrer">PUCT</a> not MCTS, since pure MCTS uses random playouts for evaluation, while Lc0 uses neural network based evaluation function. For pure MCTS depth has not a lot of meaning, since better heuristic for accuracy is number of random playouts.</p>
  <p>Another thing that makes comparision between Lc0 and Stockfish harder is difference in hardware. Lc0 runs on GPU, Stockfish on CPU.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-09T22:38:28.293Z
Answers.Text: I would suggest adding as information that for MCTS-based engines like LC0, the notion of calculation depth doesn't make much sense (this is to a lesser extent also true for modern Alpha-Beta based engines, as the intuition that they perform something remotely equivalent to full minimax search within some fixed depth horizon is very wrong; but at least, the notion of depth has some historical continuity going for it there).
Answers.UserDisplayName: Polytropos
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-14T20:04:46.490Z
Answers.Text: MCTS implementations with policy networks such as used in Leela Zero (Chess) have a much more selective search (**less** breadth) than alpha-beta based engines.
  
  PUCT is a version of UCT with Priors (given by the policy network), and UCT is just a formula to steer a MCTS. So Leela certainly still uses MCTS, it accelerates the playouts by estimating the result of them with a neural network. The MCTS doesn't "know" we optimized away the playouts, so it's not suddenly another algorithm.
Answers.UserDisplayName: gcp
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-09T13:55:30.180Z
Answers.LastActivityDate: 2023-01-10T09:06:46.263Z
Answers.LastEditDate: 2023-01-10T09:06:46.263Z
Answers.UserDisplayName: Matija Sirk
Body: <p>I'm looking for some <strong>reference</strong> for the number of lookahead steps typically used by chess agents (Stockfish / Leela Chess Zero / others?)</p>
  <p>From a quick search I found that:
  the answer depends on:</p>
  <ol>
  <li>complexity of position,</li>
  <li>computer hardware/processor speed,</li>
  <li>efficiency of software/graphics interface,</li>
  <li>time given to the chess engine.</li>
  </ol>
  <p>with somewhat mediocre hardware a <strong>depth around 15 ply's in one second</strong> is typical.</p>
  <p>the NPS value for the Stockfish engine: 49,473 kN/s (calculating 49,473,000 positions (nodes) per second).</p>
  <p>source:</p>
  <ol>
  <li><a href="https://www.quora.com/What-is-Stockfishs-depth-when-analysing-a-chess-position" rel="nofollow noreferrer">https://www.quora.com/What-is-Stockfishs-depth-when-analysing-a-chess-position</a></li>
  <li><a href="https://chessify.me/blog/nps-what-are-the-nodes-per-second-in-chess-engine-analysis" rel="nofollow noreferrer">https://chessify.me/blog/nps-what-are-the-nodes-per-second-in-chess-engine-analysis</a></li>
  </ol>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-12-20T16:31:33.353Z
Comments.Text: Seems like you already have the answer, what is the question then?
Comments.UserDisplayName: Allure
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-12-20T19:02:13.693Z
Comments.Text: I am looking for a valid reference, academic work would be best. Also I am not sure about the answer I got and I would like to get a validation.
Comments.UserDisplayName: Cohensius
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-12-21T00:18:25.027Z
Comments.Text: I can confirm the answers are right, but I can't give you a reference, because it's sort of common knowledge among the chess engine community.
Comments.UserDisplayName: Allure
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-03-14T20:10:46.883Z
Comments.Text: If you need a citable academic reference for the lookahead steps of chess engines on typical hardware, this sounds a bit like an XY problem because the answer is largely meaningless/not very well defined.
Comments.UserDisplayName: gcp
Tags: <engines><software><stockfish><leela>
Title: What is the Depth reached by chess-AI agents on a regular computer?
ThreadText: # What is the Depth reached by chess-AI agents on a regular computer?
  
  (Asked by: username_0 on 2022-12-20)
  
  I'm looking for some **reference** for the number of lookahead steps typically used by chess agents (Stockfish / Leela Chess Zero / others?)
  
  From a quick search I found that:
  the answer depends on:
  
  1. complexity of position,
  2. computer hardware/processor speed,
  3. efficiency of software/graphics interface,
  4. time given to the chess engine.
  
  with somewhat mediocre hardware a **depth around 15 ply's in one second** is typical.
  
  the NPS value for the Stockfish engine: 49,473 kN/s (calculating 49,473,000 positions (nodes) per second).
  
  source:
  
  1. <https://www.quora.com/What-is-Stockfishs-depth-when-analysing-a-chess-position>
  2. <https://chessify.me/blog/nps-what-are-the-nodes-per-second-in-chess-engine-analysis>
  
  **Tags:** engines, software, stockfish, leela
  
  > **Comments:**
  > **username_1**: Seems like you already have the answer, what is the question then?  
  > **username_0**: I am looking for a valid reference, academic work would be best. Also I am not sure about the answer I got and I would like to get a validation.  
  > **username_1**: I can confirm the answers are right, but I can't give you a reference, because it's sort of common knowledge among the chess engine community.  
  > **username_2**: If you need a citable academic reference for the lookahead steps of chess engines on typical hardware, this sounds a bit like an XY problem because the answer is largely meaningless/not very well defined.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2023-01-09. Score: 5 | Accepted answer
  
  Up to date reference is practically impossible to give, since engines and hardware change on yearly basis.
  
  You identified most important conditions, but I would like to add two more:
  
  1. Presence of tablebases (vastly improves evaluation depth in late endgames);
  2. [NNUE](https://www.chessprogramming.org/NNUE) (slower) or hand-tuned evaluation function (faster).
  
  Introduction of NNUE techniques vastly improved accuracy of evaluation function, but also made it slower. If you compare stockfish versions before NNUE and after you will see in same time frame difference of as many as 10-20 ply. Newest stockfish is still stronger than older, because it gains more in evaluation accuracy than loses in depth (in most positions! in some endgames this tradeoff is not so good).
  
  From my experience (running time 1-2 minutes):
  
  * SF before NNUE in middlegame on reasonably strong computer: 30-40 ply
  * SF with NNUE in middlegame on reasonably strong computer: 20 ply
  * SF with NNUE in middlegame with 64 cores and 128gb ram: 30-40 ply
  * SF with 6 piece tablebases in endgame with ~8 pieces: 60+ ply or (more often) forced win/draw found.
  
  Note that in early openings engines are very unreliable, no matter the (contemporarily achievable) depth.
  
  ---
  
  As [@username_4](https://chess.stackexchange.com/users/27564/username_4) pointed out in comment, meaning of depth depends on choice of tree search algorithm. Possible moves and possible replies to them (and replies to them ...) can be arrayed into tree structure, so question of finding the best move is question of finding route to victory. Since this tree is too big to cover entirely we need to cheat - stop searching before we find the goal, evaluate positions and calculate approximate scores.
  
  Additionaly chess is a zero sum game - this means, that your position can only be as good for you as it is bad for your opponent (and vice versa). This means that [minimax](https://www.chessprogramming.org/Alpha-Beta) search can be used.
  
  Detailed explanation of tree search algorithms is out of scope for this answer. If interested, consult pages for [minimax](https://www.chessprogramming.org/Alpha-Beta), [Alpha-Beta pruning](https://www.chessprogramming.org/Alpha-Beta) and [Monte-Carlo tree search (MCTS)](https://www.chessprogramming.org/Monte-Carlo_Tree_Search). I will focus on practical consequences of choice of algorithm. Theoretically given infinite time they all lead to optimal solution.
  
  Minimax in pure form cannot be used practically, since it's too time intensive. So in practice it's optimized by Alpha-Beta pruning. This way of searching is used by [Stockfish](https://stockfishchess.org/), currently strongest publically available engine. **Depth here means number of plys before reaching evaluation, with some moves ignored at each ply based on preliminary evaluations.** Practically it optimizes for greater depth by sacrificing some breadth.
  
  Consequence is that it may miss some good moves, which look bad to preliminary evaluation. Consequently human skill still matters in engine-assisted correspondence chess, since humans can force engine to consider moves that it would disregard to early (most commonly sacrifices with long-term positional compensation). It also leads to engines being weaker in (at least quiet) openings compared to (at least forcing) middlegames, since breadth is larger in openings, so more of it has to be pruned.
  
  MCTS is used most notably by AlphaZero and [Lc0](https://lczero.org/). It's based on probabilistic methods. Practically it leads to lower search depths but better breadth than minimax with Alpha-Beta pruning.
  
  Consequently play of these engines is slightly better in openings, but slightly worse in endgames (but this may also be attributed to evaluation function).
  
  **Depth for Lc0 is average depth reached for move,** based on considered sample.
  
  **In short, don't compare depths reached by one engine to depths reached by different engine. Depth is inaccurate heuristic for accuracy, as such it can only tell you, that if same engine reaches depth 20 and depth 40, evaluation at depth 40 is most probably more accurate.**
  
  ---
  
  More pedantically Lc0 uses [PUCT](https://lczero.org/dev/wiki/technical-explanation-of-leela-chess-zero/) not MCTS, since pure MCTS uses random playouts for evaluation, while Lc0 uses neural network based evaluation function. For pure MCTS depth has not a lot of meaning, since better heuristic for accuracy is number of random playouts.
  
  Another thing that makes comparision between Lc0 and Stockfish harder is difference in hardware. Lc0 runs on GPU, Stockfish on CPU.
  
  > **Comments:**
  > **username_4**: I would suggest adding as information that for MCTS-based engines like LC0, the notion of calculation depth doesn't make much sense (this is to a lesser extent also true for modern Alpha-Beta based engines, as the intuition that they perform something remotely equivalent to full minimax search within some fixed depth horizon is very wrong; but at least, the notion of depth has some historical continuity going for it there).  
  > **username_2**: MCTS implementations with policy networks such as used in Leela Zero (Chess) have a much more selective search (**less** breadth) than alpha-beta based engines.  PUCT is a version of UCT with Priors (given by the policy network), and UCT is just a formula to steer a MCTS. So Leela certainly still uses MCTS, it accelerates the playouts by estimating the result of them with a neural network. The MCTS doesn't "know" we optimized away the playouts, so it's not suddenly another algorithm.  
  
  ---
  
  