Answers.Body: <blockquote>
    <p>How does the algorithm work?  </p>
  </blockquote>
  
  <p>Starting here, the rest become trivial extensions.</p>
  
  <p>The algorithm runs under a tree structure. For example, it looks at Move A, find every corresponding move from the opponent. Then it finds all moves from yourself, over and over.  We list all of these plausible positions a score.  The process Balances itself out so in each case the algorithm spends most of it's time following branches of the tree that seem most likely to win, rather than obviously losing branches.  The score is the key here - I have run under the assumption that it is not directly decided by position (although I could be wrong now), but based upon the material value of future positions.</p>
  
  <p>As an example, if you make move A now, in a few more moves, you might have 200 plausible positions where you're up a rook, 250 where you're down a rook, 300 plausible positions where you're up a knight, 250 where you're down, 1000 where you're up a pawn, and 500 where you're down a pawn, as determined by the algorithm playing a branch.  Then it just averages the material, divided by the total number of positions):</p>
  
  <p>(200*(5) + 250*(-5) + 300*(3) + 250*(-3) + 1000*(1) + 500*(-1))/2500 = 400/2500 = 0.16.</p>
  
  <p>The branch is assigned that value.  But in the same branch of that tree, a slightly different course might get a score of 0.56.  We accumulate a bunch of scores by looking in depth 5,6,7 (in the endgame way more) moves ahead this way for all the branches with this many moves, then average all of them and get a score for move A.  We use this logic of assigning scores based upon future material gains for each of the possible moves available, until a set amount of time is up, or we've thought through a set number of moves ahead.</p>
  
  <p>Finally, if it finds a branch that would make all branches lead to a capture of the king (which is in itself mate), then it would follow this as such with a mate in X moves.  From playing with the machine, it seems the computer has arbitrarily set the king as worth 100 points, rather than the infinite game-ending value it really has.  This simply means it's worth more than the cumulative sum of the other pieces though, so has the desired effect.  </p>
  
  <p>With this knowledge, we can then answer:</p>
  
  <blockquote>
    <p>Is this correct:</p>
  </blockquote>
  
  <p>Yes.  Going a single move ahead that has a 100% chance for being up a single pawn, a +1.0 would be the algorithm's response.  A single move for black which has a 50% chance for being up a single pawn would result in an outcome from the algorithm of -0.5</p>
  
  <blockquote>
    <p>How would you scale the outputs to be human interpret-able:</p>
  </blockquote>
  
  <p>To me, a better output would be % chance of win.  However, while computers are beating humans, they haven't beaten the game.  Just to much math.  Knowing that move A gives me 4% more chance to win matters more to me then pawns I might win in the future.  The computer could only spit out based upon plausible positions where wins and losses were detected, but long term strategy usually isn't to aim for repeated Scholar's Mates.</p>
  
  <blockquote>
    <p>How would you define the loss function? </p>
  </blockquote>
  
  <p>It's the probability of having more material in the future.  A +0.01 means you have 1% chance of being up one pawn in the future if you made the intended move.</p>
  
  <blockquote>
    <p>How to test for the changes?</p>
  </blockquote>
  
  <p>This needs clarification.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-04-07T16:13:38.513Z
Answers.LastActivityDate: 2017-04-07T17:30:23.413Z
Answers.LastEditDate: 2017-04-07T17:30:23.413Z
Answers.UserDisplayName: Mark
Answers.Body: <p>Well, I found a place where a member of the Stockfish support staff <a href="https://web.archive.org/web/20180424122041/http://support.stockfishchess.org/discussions/questions/1303-stockfish-evaluation-numbers-and-their-meaning" rel="nofollow noreferrer">says</a>:</p>
  <blockquote>
  <p>Your intuition is correct. Positive values mean white is winning.
  Negative values mean black is winning. The units are in pawn units, so
  +1.00 means white is winning by one pawn.</p>
  </blockquote>
  <p>So, it seems that they at least claim it corresponds.  I don't know how exact it is; there really isn't an &quot;exact&quot; value for any piece since it depends on the position anyway.  But I think that's your answer.</p>
  <p>You say:</p>
  <blockquote>
  <p>We only care if the optimization can give the engine gradients to a
  local optimum solution.</p>
  </blockquote>
  <p>I disagree.  Your premise that Stockfish is only used to find the best move is incorrect - some of us actually look at the evaluation score.  If I'm using Stockfish to look at a game I played, I absolutely care whether the move I played lost me 5 pawns or 0.05 pawns compared to the best move (one is a blunder, the other is a perfectly reasonable move and might not even be objectively worse.)  And it makes a difference whether a move took the evaluation from -10 pawns to -20 (it was completely hopeless anyway and I shouldn't worry too much), or -1 pawns to -2 (going from disadvantaged to probably lost, and maybe I should study that a bit more to see what went wrong.)</p>
  <p>Yes, you can scale it however you like - and since you can, why not just scale it so a pawn is 1?</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-04-07T06:15:18.837Z
Answers.Text: If my answer deserves a downvote, perhaps I didn't understand the question.
Answers.UserDisplayName: D M
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-04-07T08:02:22.450Z
Answers.Text: "why not just scale it so a pawn is 1" -- note that that's not trivial at all: _which_ pawn, in _which_ position?
Answers.UserDisplayName: RemcoGerlich
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-04-07T11:47:27.390Z
Answers.Text: @RemcoGerlich is correct. In my question, I didn't rule out the possibility of scaling to something human interpretable (e.g Houdini claims the engine is made for that). But the point is ... Stockfish is not made for that. Morever, scaling to a pawn advantage of 1.0 is actually very hard. How to scale? What algorithm to scale?
Answers.UserDisplayName: SmallChess
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-04-07T18:15:37.130Z
Answers.Text: @RemcoGerlich I did mention that it depends on the position.  But taking the e-pawn from White in the opening position took it from +0.3 to -0.7 at a depth of 19, and taking the e-pawn from Black took it from +0.3 to +1.3.  I used lichess.org, which uses Stockfish.
Answers.UserDisplayName: D M
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-04-07T18:30:37.783Z
Answers.Text: @SmallChess Well, as I pointed out in my answer, the Stockfish support staff said it IS made for that.  Why would you think it's true for Houdini and not Stockfish, given they both claim the same thing?
Answers.UserDisplayName: D M
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2017-04-06T23:19:14.300Z
Answers.LastActivityDate: 2022-12-25T17:48:48.423Z
Answers.LastEditDate: 2022-12-25T17:48:48.423Z
Answers.UserDisplayName: D M
Answers.Body: <blockquote>
    <p>In fact, we should be able to scale all all the PV scores by a constant. As long as the engine makes the same move, the magnitude of the scores shouldn't matter.</p>
  </blockquote>
  
  <p>True. We can even add a constant to all scores (e.g. an equal position would have a score of +1) without any effect on the engine's decisions.</p>
  
  <p>However, it's useful to keep the well-known score definitions:</p>
  
  <blockquote>
    <p>+1.0 is a pawn advantage for White</p>
  </blockquote>
  
  <p>for 'backwards compatibility', just like we still have a <a href="https://en.wikipedia.org/wiki/History_of_the_metre#International_prototype_metre" rel="nofollow noreferrer">'meter bar' in Paris</a> while the meter is currently defined with <a href="https://en.wikipedia.org/wiki/Isotopes_of_krypton#Krypton-86" rel="nofollow noreferrer">Krypton 86</a>, so the 'meter bar' is technically slightly longer or shorter than a meter.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-04-06T13:48:40.353Z
Answers.Text: We all known +1.0 **should be** a pawn advantage. I'm not saying NO. My point is that the outputs from Stockfish **are not made** for that. Houdini author claims his engine is made for that, but not Stockfish. I also believe (but not 100% sure) Komodo claims for the +1.0 pawn interpretation.
Answers.UserDisplayName: SmallChess
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-04-06T13:50:39.900Z
Answers.Text: If Stockfish **are not made for that**, should we still tell everybody "+1.0 is a pawn advantage for White in Stockfish"??
Answers.UserDisplayName: SmallChess
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-04-06T13:53:51.707Z
Answers.Text: There's nobody stopping you from using centipawns as your scoring unit. It's just used less often.
Answers.UserDisplayName: Glorfindel
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-04-06T13:55:51.827Z
Answers.Text: Even if we were to use centipawns, that still doesn't solve the problem. Can we say "100 centipawn output from Stockfish mean one pawn advantage?" Sorry, I'm just here to start a potentially interesting discussion. I'm sure some users will find our discussion interesting.
Answers.UserDisplayName: SmallChess
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-04-06T13:57:20.870Z
Answers.Text: Yes. The error in that is much smaller than the tendency of even grandmasters to make inferior moves in a certain position. E.g. nobody sees the difference between a +0.8 and a +0.9 position.
Answers.UserDisplayName: Glorfindel
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-04-06T13:47:18.383Z
Answers.LastActivityDate: 2017-04-06T13:47:18.383Z
Answers.UserDisplayName: Glorfindel
Body: <p>Recent question:</p>
  
  <blockquote>
    <p><a href="https://chess.stackexchange.com/questions/17192/how-are-the-scores-decided-in-chess">How are the scores decided in Chess?</a></p>
  </blockquote>
  
  <p>It has been argued that PV scores for Stockfish (the engine being used by chess.com) can be interpreted as something like:</p>
  
  <blockquote>
    <p>+1.0 is a pawn advantage for White. -0.5 is a half-pawn advantage for Black. </p>
  </blockquote>
  
  <p><strong>Is that correct?</strong> The weights and parameters in Stockfish are either manually calibrated or tuned by something like the <a href="https://chessprogramming.wikispaces.com/SPSA" rel="nofollow noreferrer">SPSA optimiser</a>. We only care if the optimization can give the engine gradients to a local optimum solution.</p>
  
  <p>The objective function is to maximise the playing strength. Maximising playing strength has nothing to do with making sure "+1.0 is a pawn advantage for White". In fact, we should be able to <strong>scale</strong> all all the PV scores by a constant. As long as the engine makes the same move, the <strong>magnitude</strong> of the scores shouldn't matter.</p>
  
  <p>This is consistent to the minimax algorithm, where both players <strong>only care</strong> how to minimize/maximise the evaluation. Again, there are <strong>no efforts</strong> to make the PV scores human-interpretable.</p>
  
  <p>To me, the PV scores are only important <strong>relatively</strong>, not <strong>absolutely</strong>. For instance, we should be able to say "+2.0 is about two-times advantage for White than a position with +1.0 for White.", but that <strong>shouldn't</strong> have anything to do with pawns. Similarly, if I modify the source code and scale the outputs by a factor of 10. I can still say: "+20.0 is about two-times advantage for White than a position with +10.0 for White". We <strong>shouldn't</strong> be able to connect the scores to pawns, knights, bishops etc.</p>
  
  <p>I haven't seen anybody in the Stockfish development team spent any efforts to make the outputs interpretable. To me, the scores have <strong>no direct interpretation</strong> to chess.</p>
  
  <p>Should I call all answers in the question <strong>technically</strong> incorrect?</p>
  
  <p><strong>EDIT</strong></p>
  
  <p>This is in response to @D_M. How would you scale the outputs to make it human interpretable? What's the algorithm? How would you define the loss function? How to test for the changes? I don't see it's a trivial problem.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-03T18:40:46.113Z
Comments.Text: Note that Stockfish 15.1, released in December 2022 five years after this question was asked, [rescaled](https://stockfishchess.org/blog/2022/stockfish-15-1/) evaluations significantly.
Comments.UserDisplayName: Federico Poloni
Tags: <engines>
Title: "How are the scores decided in Chess?" - Correct?
ThreadText: # "How are the scores decided in Chess?" - Correct?
  
  (Asked by: username_0 on 2017-04-06)
  
  Recent question:
  
  > [How are the scores decided in Chess?](https://chess.stackexchange.com/questions/17192/how-are-the-scores-decided-in-chess)
  
  It has been argued that PV scores for Stockfish (the engine being used by chess.com) can be interpreted as something like:
  
  > +1.0 is a pawn advantage for White. -0.5 is a half-pawn advantage for Black.
  
  **Is that correct?** The weights and parameters in Stockfish are either manually calibrated or tuned by something like the [SPSA optimiser](https://chessprogramming.wikispaces.com/SPSA). We only care if the optimization can give the engine gradients to a local optimum solution.
  
  The objective function is to maximise the playing strength. Maximising playing strength has nothing to do with making sure "+1.0 is a pawn advantage for White". In fact, we should be able to **scale** all all the PV scores by a constant. As long as the engine makes the same move, the **magnitude** of the scores shouldn't matter.
  
  This is consistent to the minimax algorithm, where both players **only care** how to minimize/maximise the evaluation. Again, there are **no efforts** to make the PV scores human-interpretable.
  
  To me, the PV scores are only important **relatively**, not **absolutely**. For instance, we should be able to say "+2.0 is about two-times advantage for White than a position with +1.0 for White.", but that **shouldn't** have anything to do with pawns. Similarly, if I modify the source code and scale the outputs by a factor of 10. I can still say: "+20.0 is about two-times advantage for White than a position with +10.0 for White". We **shouldn't** be able to connect the scores to pawns, knights, bishops etc.
  
  I haven't seen anybody in the Stockfish development team spent any efforts to make the outputs interpretable. To me, the scores have **no direct interpretation** to chess.
  
  Should I call all answers in the question **technically** incorrect?
  
  **EDIT**
  
  This is in response to @D\_M. How would you scale the outputs to make it human interpretable? What's the algorithm? How would you define the loss function? How to test for the changes? I don't see it's a trivial problem.
  
  **Tags:** engines
  
  > **Comments:**
  > **username_1**: Note that Stockfish 15.1, released in December 2022 five years after this question was asked, [rescaled](https://stockfishchess.org/blog/2022/stockfish-15-1/) evaluations significantly.  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_2 on 2017-04-07. Score: 2 | Accepted answer
  
  > How does the algorithm work?
  
  Starting here, the rest become trivial extensions.
  
  The algorithm runs under a tree structure. For example, it looks at Move A, find every corresponding move from the opponent. Then it finds all moves from yourself, over and over. We list all of these plausible positions a score. The process Balances itself out so in each case the algorithm spends most of it's time following branches of the tree that seem most likely to win, rather than obviously losing branches. The score is the key here - I have run under the assumption that it is not directly decided by position (although I could be wrong now), but based upon the material value of future positions.
  
  As an example, if you make move A now, in a few more moves, you might have 200 plausible positions where you're up a rook, 250 where you're down a rook, 300 plausible positions where you're up a knight, 250 where you're down, 1000 where you're up a pawn, and 500 where you're down a pawn, as determined by the algorithm playing a branch. Then it just averages the material, divided by the total number of positions):
  
  (200\*(5) + 250\*(-5) + 300\*(3) + 250\*(-3) + 1000\*(1) + 500\*(-1))/2500 = 400/2500 = 0.16.
  
  The branch is assigned that value. But in the same branch of that tree, a slightly different course might get a score of 0.56. We accumulate a bunch of scores by looking in depth 5,6,7 (in the endgame way more) moves ahead this way for all the branches with this many moves, then average all of them and get a score for move A. We use this logic of assigning scores based upon future material gains for each of the possible moves available, until a set amount of time is up, or we've thought through a set number of moves ahead.
  
  Finally, if it finds a branch that would make all branches lead to a capture of the king (which is in itself mate), then it would follow this as such with a mate in X moves. From playing with the machine, it seems the computer has arbitrarily set the king as worth 100 points, rather than the infinite game-ending value it really has. This simply means it's worth more than the cumulative sum of the other pieces though, so has the desired effect.
  
  With this knowledge, we can then answer:
  
  > Is this correct:
  
  Yes. Going a single move ahead that has a 100% chance for being up a single pawn, a +1.0 would be the algorithm's response. A single move for black which has a 50% chance for being up a single pawn would result in an outcome from the algorithm of -0.5
  
  > How would you scale the outputs to be human interpret-able:
  
  To me, a better output would be % chance of win. However, while computers are beating humans, they haven't beaten the game. Just to much math. Knowing that move A gives me 4% more chance to win matters more to me then pawns I might win in the future. The computer could only spit out based upon plausible positions where wins and losses were detected, but long term strategy usually isn't to aim for repeated Scholar's Mates.
  
  > How would you define the loss function?
  
  It's the probability of having more material in the future. A +0.01 means you have 1% chance of being up one pawn in the future if you made the intended move.
  
  > How to test for the changes?
  
  This needs clarification.
  
  
  ---
  
  ### Answer by username_3 on 2017-04-06. Score: 2
  
  Well, I found a place where a member of the Stockfish support staff [says](https://web.archive.org/web/20180424122041/http://support.stockfishchess.org/discussions/questions/1303-stockfish-evaluation-numbers-and-their-meaning):
  
  > Your intuition is correct. Positive values mean white is winning.
  > Negative values mean black is winning. The units are in pawn units, so
  > +1.00 means white is winning by one pawn.
  
  So, it seems that they at least claim it corresponds. I don't know how exact it is; there really isn't an "exact" value for any piece since it depends on the position anyway. But I think that's your answer.
  
  You say:
  
  > We only care if the optimization can give the engine gradients to a
  > local optimum solution.
  
  I disagree. Your premise that Stockfish is only used to find the best move is incorrect - some of us actually look at the evaluation score. If I'm using Stockfish to look at a game I played, I absolutely care whether the move I played lost me 5 pawns or 0.05 pawns compared to the best move (one is a blunder, the other is a perfectly reasonable move and might not even be objectively worse.) And it makes a difference whether a move took the evaluation from -10 pawns to -20 (it was completely hopeless anyway and I shouldn't worry too much), or -1 pawns to -2 (going from disadvantaged to probably lost, and maybe I should study that a bit more to see what went wrong.)
  
  Yes, you can scale it however you like - and since you can, why not just scale it so a pawn is 1?
  
  > **Comments:**
  > **username_3**: If my answer deserves a downvote, perhaps I didn't understand the question.  
  > **username_4**: "why not just scale it so a pawn is 1" -- note that that's not trivial at all: _which_ pawn, in _which_ position?  
  > **username_0**: @username_4 is correct. In my question, I didn't rule out the possibility of scaling to something human interpretable (e.g Houdini claims the engine is made for that). But the point is ... Stockfish is not made for that. Morever, scaling to a pawn advantage of 1.0 is actually very hard. How to scale? What algorithm to scale?  
  > **username_3**: @username_4 I did mention that it depends on the position.  But taking the e-pawn from White in the opening position took it from +0.3 to -0.7 at a depth of 19, and taking the e-pawn from Black took it from +0.3 to +1.3.  I used lichess.org, which uses Stockfish.  
  > **username_3**: @username_0 Well, as I pointed out in my answer, the Stockfish support staff said it IS made for that.  Why would you think it's true for Houdini and not Stockfish, given they both claim the same thing?  
  
  ---
  
  ### Answer by username_5 on 2017-04-06. Score: 1
  
  > In fact, we should be able to scale all all the PV scores by a constant. As long as the engine makes the same move, the magnitude of the scores shouldn't matter.
  
  True. We can even add a constant to all scores (e.g. an equal position would have a score of +1) without any effect on the engine's decisions.
  
  However, it's useful to keep the well-known score definitions:
  
  > +1.0 is a pawn advantage for White
  
  for 'backwards compatibility', just like we still have a ['meter bar' in Paris](https://en.wikipedia.org/wiki/History_of_the_metre#International_prototype_metre) while the meter is currently defined with [Krypton 86](https://en.wikipedia.org/wiki/Isotopes_of_krypton#Krypton-86), so the 'meter bar' is technically slightly longer or shorter than a meter.
  
  > **Comments:**
  > **username_0**: We all known +1.0 **should be** a pawn advantage. I'm not saying NO. My point is that the outputs from Stockfish **are not made** for that. Houdini author claims his engine is made for that, but not Stockfish. I also believe (but not 100% sure) Komodo claims for the +1.0 pawn interpretation.  
  > **username_0**: If Stockfish **are not made for that**, should we still tell everybody "+1.0 is a pawn advantage for White in Stockfish"??  
  > **username_5**: There's nobody stopping you from using centipawns as your scoring unit. It's just used less often.  
  > **username_0**: Even if we were to use centipawns, that still doesn't solve the problem. Can we say "100 centipawn output from Stockfish mean one pawn advantage?" Sorry, I'm just here to start a potentially interesting discussion. I'm sure some users will find our discussion interesting.  
  > **username_5**: Yes. The error in that is much smaller than the tendency of even grandmasters to make inferior moves in a certain position. E.g. nobody sees the difference between a +0.8 and a +0.9 position.  
  
  ---
  
  