Answers.Body: <p>Run a shallow search (you might want to add multi-pv). Check the magnuitude of the lines if you want to re-run a proper search. Although this sounds weird, but this is a very common strategy in Stockfish (eg: move reductions). Your won't waste your previous search because the hash-table should speed up your next search. You can read more <a href="https://chessprogramming.wikispaces.com/Hash+Table" rel="nofollow">here</a>.</p>
  
  <p>For example, you wouldn't want to waste CPU cycles in a position where white is a rook ahead.</p>
  
  <p>Otherwise, I don't see a non-trivial way to detect if a position is "interesting" to search.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2015-10-19T23:50:53.707Z
Answers.LastActivityDate: 2015-10-19T23:50:53.707Z
Answers.UserDisplayName: SmallChess
Body: <p>I would like to have Stockfish (or another strong engine) automatically analyze a very large database of games (>1M games), so that the analysis is as strong as possible while making the best use of my computational resources. A naive approach would be to run Stockfish on every position in every game to a certain depth, and then report the evaluation. But this would waste CPU cycles on calculating trivial positions for way too long, for example positions in which a simple recapture is obviously the only not-losing move. How should I determine how long I should let the engine think in each position?</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2015-10-19T23:26:09.913Z
Comments.Text: Can you run a shadow search and check the magnitude of the evaluation? If it's over a certain threshold, run a deeper analysis. The hash-table should help you in the next search.
Comments.UserDisplayName: SmallChess
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2015-10-19T23:26:41.263Z
Comments.Text: Also, you might want to run the analysis across a cluster.
Comments.UserDisplayName: SmallChess
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2015-10-19T23:37:59.360Z
Comments.Text: Hi @StudentT, thanks for the comments. Yes, I intend to run this across a cluster. What do you mean by a shadow search? And how can I use the hash-table in the next search?
Comments.UserDisplayName: Big Dogg
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2015-10-19T23:40:38.440Z
Comments.Text: Sorry, I meant shallow search. I misspelled. I'll type a proper answer.
Comments.UserDisplayName: SmallChess
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2015-10-20T08:07:23.720Z
Comments.Text: You might want to consider merging the games into a variation tree before you start evaluating positions. Otherwise you will invariably evaluate the same opening lines many times.
Comments.UserDisplayName: BlindKungFuMaster
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2015-10-20T08:08:19.987Z
Comments.Text: Also, how many CPU hours do you intend to invest? And what is the minimum strength you could still accept?
Comments.UserDisplayName: BlindKungFuMaster
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2015-10-20T15:50:10.690Z
Comments.Text: Hi @BlindKungFuMaster, that's an interesting idea. I was originally going to only analyze positions that are unique to avoid opening lines, but perhaps I could do something along the lines you mention. I would like a strength at least 3000. I'm willing to invest pretty much as many CPU hours as needed... 10s of thousands at least.
Comments.UserDisplayName: Big Dogg
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2015-10-20T15:55:28.960Z
Comments.Text: The problem is that you are already in 10000s cpu hours if you invest just one second per position. And that is nowhere close to 3000 Elo. To get an idea of the playing strength of engines /depth (which you have to transfer somehow into strength/time) take a look at this question: http://chess.stackexchange.com/questions/6565/what-is-the-elo-rating-of-a-chess-computer-that-can-only-look-1-2-3-moves/8956#8956
Comments.UserDisplayName: BlindKungFuMaster
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2015-10-20T16:46:24.980Z
Comments.Text: Yes sorry, I meant at least 10000s CPU hours for a subset of the 1M games. I'd rather do a smaller number of games well than a large number of games poorly... the strength of the evaluations is paramount.
Comments.UserDisplayName: Big Dogg
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2015-10-21T21:24:09.890Z
Comments.Text: @BlindKungFuMaster If I want the maximum strength evaluations on moves in the middlegame (I'll probably exclude positions from the opening that have been before and endgame positions that aren't in tablebases), would it better to run each position to a certain depth, or give each position the same fixed amount of CPU time? [assume these two approaches are calibrated to use the same total amount of time, they just differ in how they distribute cpu time across moves]
Comments.UserDisplayName: Big Dogg
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2015-10-22T07:32:48.043Z
Comments.Text: I would say time is more sensible to specify. In simple positions you often have to (and are able to) calculate much deeper than in complicated positions. Or to put it differently: If you specify a search depth you might lose 5 ply in simple positions just to gain 1 ply in complicated positions. But: Engines have heuristics to decide how long to think with a given time control in a given position. You should probably tap into this for an optimal result. Chances are the engine nows a lot better how to manage its time, than you could externally decide.
Comments.UserDisplayName: BlindKungFuMaster
Tags: <analysis><programming>
Title: How to use Stockfish to automatically analyze big DB of games
ThreadText: # How to use Stockfish to automatically analyze big DB of games
  
  (Asked by: username_0 on 2015-10-19)
  
  I would like to have Stockfish (or another strong engine) automatically analyze a very large database of games (>1M games), so that the analysis is as strong as possible while making the best use of my computational resources. A naive approach would be to run Stockfish on every position in every game to a certain depth, and then report the evaluation. But this would waste CPU cycles on calculating trivial positions for way too long, for example positions in which a simple recapture is obviously the only not-losing move. How should I determine how long I should let the engine think in each position?
  
  **Tags:** analysis, programming
  
  > **Comments:**
  > **username_1**: Can you run a shadow search and check the magnitude of the evaluation? If it's over a certain threshold, run a deeper analysis. The hash-table should help you in the next search.  
  > **username_1**: Also, you might want to run the analysis across a cluster.  
  > **username_0**: Hi @StudentT, thanks for the comments. Yes, I intend to run this across a cluster. What do you mean by a shadow search? And how can I use the hash-table in the next search?  
  > **username_1**: Sorry, I meant shallow search. I misspelled. I'll type a proper answer.  
  > **username_2**: You might want to consider merging the games into a variation tree before you start evaluating positions. Otherwise you will invariably evaluate the same opening lines many times.  
  > **username_2**: Also, how many CPU hours do you intend to invest? And what is the minimum strength you could still accept?  
  > **username_0**: Hi @username_2, that's an interesting idea. I was originally going to only analyze positions that are unique to avoid opening lines, but perhaps I could do something along the lines you mention. I would like a strength at least 3000. I'm willing to invest pretty much as many CPU hours as needed... 10s of thousands at least.  
  > **username_2**: The problem is that you are already in 10000s cpu hours if you invest just one second per position. And that is nowhere close to 3000 Elo. To get an idea of the playing strength of engines /depth (which you have to transfer somehow into strength/time) take a look at this question: http://chess.stackexchange.com/questions/6565/what-is-the-elo-rating-of-a-chess-computer-that-can-only-look-1-2-3-moves/8956#8956  
  > **username_0**: Yes sorry, I meant at least 10000s CPU hours for a subset of the 1M games. I'd rather do a smaller number of games well than a large number of games poorly... the strength of the evaluations is paramount.  
  > **username_0**: @username_2 If I want the maximum strength evaluations on moves in the middlegame (I'll probably exclude positions from the opening that have been before and endgame positions that aren't in tablebases), would it better to run each position to a certain depth, or give each position the same fixed amount of CPU time? [assume these two approaches are calibrated to use the same total amount of time, they just differ in how they distribute cpu time across moves]  
  > **username_2**: I would say time is more sensible to specify. In simple positions you often have to (and are able to) calculate much deeper than in complicated positions. Or to put it differently: If you specify a search depth you might lose 5 ply in simple positions just to gain 1 ply in complicated positions. But: Engines have heuristics to decide how long to think with a given time control in a given position. You should probably tap into this for an optimal result. Chances are the engine nows a lot better how to manage its time, than you could externally decide.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2015-10-19. Score: 2
  
  Run a shallow search (you might want to add multi-pv). Check the magnuitude of the lines if you want to re-run a proper search. Although this sounds weird, but this is a very common strategy in Stockfish (eg: move reductions). Your won't waste your previous search because the hash-table should speed up your next search. You can read more [here](https://chessprogramming.wikispaces.com/Hash+Table).
  
  For example, you wouldn't want to waste CPU cycles in a position where white is a rook ahead.
  
  Otherwise, I don't see a non-trivial way to detect if a position is "interesting" to search.
  
  
  ---
  
  