Answers.Body: <p>Elo is a system to summarize past results into a meaningful number that informs about the (relative) strength of a player. Since the only objectives a player has during a game of chess are getting a win or a draw, only success and failure in those two specific objectives should be taken into account. It's true that how accurately someone played in a game can be used to predict how many wins he'll get in the future, but at least if we want our rankings to have an &quot;official&quot; value (granting titles, qualifying for tournaments...), we should base those rankings on past results alone rather than trying to make predictions about potential future outcomes.</p>
  <p>But even if we want to use an engine-similarity ranking as just a way to determine who is the best player of all time (parallel to the official Elo rankings), it still comes with its share of problems:</p>
  <ul>
  <li><p>First we'd need to choose a specific engine to use. If we just take the one with the highest Elo at the moment, we're going back to taking past results as the ultimate metric of strength. We'd also need to decide of how much time per move and on what hardware the engine should run. These are again arbitrary choices that could leave to dramatically different outcomes.</p>
  </li>
  <li><p>Since new and stronger engines are coming up all the time, we'd need to be constantly recalculating past ranking lists. It'd seem unfair to dismiss &quot;Player A&quot; as the strongest player of 2024 just because the engines of the time weren't good enough to understand the mistakes of their opponents.</p>
  </li>
  <li><p>Chess has many different time controls yet FIDE does fine with just 3 different ratings (Blitz, Rapid and Classical). An accuracy-based system would require to introduce some correction for specific time controls (or to generate separate lists for 60+30, 90+30 and so on).</p>
  </li>
  <li><p>Chess is a two-player game. Forcing your opponent into making a mistake is just as important as avoiding mistakes yourself. Engines have no concept of the &quot;difficulty&quot; of a move. For instance in a theoretically drawn endgame like rook+bishop vs rook, giving the bishop away for free is just as &quot;accurate&quot; as making a challenging move that forces your opponent into a difficult defense.</p>
  </li>
  <li><p>Quiet games usually lead to easier positions where fewer mistakes are played. An accuracy-based ranking would be heavily biased against  aggressive players. Many of Tal's sacrifices are &quot;blunders&quot; even though they actually led to wins.</p>
  </li>
  <li><p>In an Elo system, you win the biggest amount of points by playing higher-rated opposition. If you just &quot;farm&quot; lower-rated players, your rating will increase very slowly. But it's much easier to get high accuracies against bad players, so you can massively inflate your score by brilliantly defeating newcomers.</p>
  </li>
  </ul>
  <p>In short, a system that rewards a player for a Berlin draw against a beginner but punishes him for a win in a chaotic double-edge position against a Grandmaster can never be a valid way to measure chess strength.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-17T13:53:37.127Z
Answers.Text: 'Engines have no concept of the "difficulty" of a move.' - I mean, some Engines definitely do have this concept.  They aren't pure search-to-solution systems.  Even against an equally smart AI, knowing that deep down a search tree there are moves/situations that are easier or harder for the other side to analyze has value for the AI.
Answers.UserDisplayName: Yakk
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-17T17:15:12.477Z
Answers.Text: @Yakk yeah but that only really applies to engine vs engine play, not really at "correcting" accuracy scores to reflect human difficulty. Even if the computer could perfectly measure how difficult a position will be for a human, the compensation for game accuracy that it'd introduce would always be arbitrary.
Answers.UserDisplayName: David
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-18T08:14:09.163Z
Answers.Text: A lot of this does seem to be based on prejudices/lacking knowledge of technical possibilities. It seems well possible to overcome some of the criticism by designing a chess engine with the goal of measuring strength instead of playing to win. Other points (who defines what is best) are very valid though.
Answers.UserDisplayName: DonQuiKong
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-18T16:06:41.080Z
Answers.Text: Larry Kaufman addresses almost all of these points in his study https://www.chess.com/article/view/chess-accuracy-ratings-goat
Answers.UserDisplayName: qwr
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-16T20:27:25.280Z
Answers.LastActivityDate: 2024-07-17T17:28:08.333Z
Answers.LastEditDate: 2024-07-17T17:28:08.333Z
Answers.UserDisplayName: David
Answers.Body: <blockquote>
  <p>Why don't we take the average rating evaluation of a player's game to measure the player's strength instead of having elo systems etc?</p>
  </blockquote>
  <p>The answer is simple and obvious. The elo system requires only the results of games and the statistical calculations are straightforward and quick. What you suggest would require all the moves of all the over-the-board FIDE  rated games be available before it could even start. Then enormous computer power would be required to analyse the hundreds of thousands of games played every month.</p>
  <p>That just applies to standard time control games. It would be completely impossible to do for the vast majority of blitz and rapid games which are not played on electronic DGT boards which record the moves. There is no record of the moves played hence rapid and blitz ratings using this system would be impossible.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-16T18:56:02.737Z
Answers.LastActivityDate: 2024-07-18T16:30:33.007Z
Answers.LastEditDate: 2024-07-18T16:30:33.007Z
Answers.UserDisplayName: Brian Towers
Answers.Body: <p>Complementary to the other more pragmatic answers let's take a step back and approach this from a more theoretical angle: The essence of a rating system is the prediction of the result of a game between two players. And since human players typically do not play deterministically, they essentially constitute or employ a mixed strategy from a game theoretical perspective. Therefore, the rating is a predictor for the statistically expected score between two such mixed strategies with only limited information about these strategies/players.</p>
  <p>Therefore you need two things to define a rating:</p>
  <ol>
  <li>Knowledge about the reference population (rating pool) of strategies.</li>
  <li>Knowledge about the strategy/player to be rated.</li>
  </ol>
  <p>What is the optimal way to use this knowledge to construct a predictor for the expected score? To me, it is not a given that only using game results is the optimal way. If you could build an approximate model of the mixed strategy and the reference strategies (e.g., using machine learning) from game records, you could do a monte-carlo simulation to give an expected score and hence a rating.</p>
  <p>All I want to say with this is that while it might practically currently not be very accurate or feasible to predict playing strength from just moves without game results, it is far from an absurd question. It is just that purely result based rating estimation has been the most transparent and robust so far, especially given the incomplete knowledge and constant changes of both the player pool and individual players.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-17T08:00:58.740Z
Answers.Text: Every rating system in some form or another works by comparing the achieved result against the expected result to be able to update rating. If there is no way to calculate an expected result, then how do you update rating other than recalculating from scratch? And a similar rating adjustment scenario to what you described happened at the beginning of this year since Elo got worse at predicting results, so FIDE seems to disagree with you. https://www.fide.com/news/2831
Answers.UserDisplayName: Fabian Fichter
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-18T02:28:56.643Z
Answers.Text: @David: Arpad Elo very explicitly constructed the Elo rating system around the idea that a player's skill level should be predictive of how many games they win and lose against other players (of varying skill levels). If a player's Elo rating does not correctly reflect the true probability of winning, they should gain or lose points until this is no longer the case (but that process might take many games).
Answers.UserDisplayName: Kevin
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-18T08:41:38.847Z
Answers.Text: Yes, the only thing that has largely remained the same the whole time is the calculation of the expected score, because that is the foundation. The update system keeps being changed just so that ratings reflect the expected scores again, i.e., are accurate/stable. But this discussion has become too long, anyone still in doubt can form his own opinion just reading from FIDE's own rating system consultant https://www.fide.com/docs/presentations/Sonas%20Proposal%20-%20Repairing%20the%20FIDE%20Standard%20Elo%20Rating%20System.pdf
Answers.UserDisplayName: Fabian Fichter
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-16T21:57:05.243Z
Answers.LastActivityDate: 2024-07-16T21:57:05.243Z
Answers.UserDisplayName: Fabian Fichter
Answers.Body: <p>Stockfish does not evaluate the rating performance of a player in a game, it can only give an opinion on the accuracy of a player's moves. I say &quot;opinion&quot; because stockfish does (rarely) lose games to other computers, so it is not 100% accurate.<br />
  At master level, certain openings can be played with high accuracy very easily. Other openings are too complicated for a human to navigate with computer precision. This does not matter to the player, because they are playing other humans. Should a player receive a higher rating because they are consistently drawing with 98% accuracy in simple openings rather than seizing advantages in messy, imperfect middlegames? I don't think so.<br />
  Some sites like chess.com offer an estimate of a player's Elo following every game. I think this is mostly a novelty and another way to &quot;gamify&quot; chess improvement. The numbers it gives should be taken with a grain of salt.</p>
  <p>EDIT: Brian Towers answer is maybe the &quot;real&quot; answer here. Even if a perfect engine was created that could also judge how difficult moves are to find from a human perspective, tournament directors don't want to have to input all moves from every game. It may seem like that's something they already do since you can find massive databases of master games. However, it's easy to forget that FIDE and USCF sanction tournaments for all skill levels. The Elo system is fast and <em><strong>practical</strong></em>, it allows a player to receive an official rating using only their tournament results.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-17T14:02:00.890Z
Answers.Text: I mean, Stockfish also provides board ratings (the black/white bar) - if you measure the impact of each player's move on that board rating, you get "more" than accuracy, in that a non-perfect move that results in nearly as good board position is different than a non-perfect move that results in a bad board position.  "The average move was -0.5 off ideal against a foe whose average move was -1.0 off ideal" could be turned into a reasonable rating system; but even here, we end up factoring in how good the opponent is, as "force the opponent to play badly" is a chess skill.
Answers.UserDisplayName: Yakk
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-17T14:45:51.277Z
Answers.Text: @Yakk it sounds like you're describing accuracy. At amateur level that is maybe a good metric for strength. Once you reach strong master level, I don't think accuracy is a good way to rank strength.
Answers.UserDisplayName: Awalrod
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-16T18:56:16.760Z
Answers.LastActivityDate: 2024-07-16T19:03:06.710Z
Answers.LastEditDate: 2024-07-16T19:03:06.710Z
Answers.UserDisplayName: Awalrod
Answers.Body: <p>If the aim is to <em><strong>find the best player of all time</strong></em> using a computer systems that looks at the moves a player makes, it is clearly not possible as explained by other answers.</p>
  <p>If we believe</p>
  <blockquote>
  <p>The essence of a rating system is the prediction of the result of a
  game between two players</p>
  </blockquote>
  <p>We can't hope to use a computer based measurement of the accuracy of a player's moves to replace rating systems, I will not repeat other answers that expain why.</p>
  <p>What what if we believe?</p>
  <blockquote>
  <p>The primary aim of a rating system to to divide players in a chess
  festival between separate Swiss Tournaments so it is likely that most
  games will be fun, eg the outcome is not predictable.</p>
  </blockquote>
  <p>And we also believe?</p>
  <blockquote>
  <p>Players who have not played in formal tournaments against anyone
  outside of their school chess club should be able to be allocated to
  the correct Switt Tournament so they get many fun games</p>
  <p>This includes when none of the club members have regularly played
  people with a reliable ratings</p>
  </blockquote>
  <p>Then we have an interesting and solvable problem with two likely solutions:</p>
  <ul>
  <li>Use a computer to get accuracy of a player's moves, then use the result as the 1st approximation of their rating.</li>
  <li>Get them to play agaist a system like <a href="https://maiachess.com/#:%7E:text=Maia%20is%20an%20AlphaZero%2FLeela,milestone%20between%201100%20and%201900." rel="nofollow noreferrer">Maia</a>, finding the level that they tend to win as many games as lose, then use the result as the 1st approximation of their rating.</li>
  </ul>
  <p>Once we have that 1st approximation of their rating and the  1st approximation of the rating ratings of other members of the schools chess club we can combine them with known recent game results from the club and feed the information into a linear programming solver to give an improved approximation of their ratings.</p>
  <p>In other words we could consider each school chess club to be a disconnected <em>rating pool</em>, and use the computer based methods to work out an approximation of the mapping function between these rating pools.</p>
  <hr />
  <p>But we could likely practically do as well by using the opponents Chess.com rating to work out the &quot;tournament rating&quot; of each person in a rating pools who played recent OTB games, then use these &quot;tournament ratings&quot; to work out the mapping between each chess clubs internal rating system.   But that is no what the question asked!</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-17T19:49:49.503Z
Answers.Text: Regarding your first sentence: there was a paper that did exactly that to determine the most accurate player of all time.
Answers.UserDisplayName: qwr
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-18T07:24:36.450Z
Answers.Text: @qwr The best players study their opponents and play differently defending on the opponents, so past play can't predict how they will play someone they never played before but greatly respects.
Answers.UserDisplayName: Ian Ringrose
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-18T16:32:48.483Z
Answers.Text: Playing style has already been incorporated into studies on historical players. In fact, it is very possible to predict play *and* validate our estimates statistically.
Answers.UserDisplayName: qwr
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-17T09:42:10.703Z
Answers.LastActivityDate: 2024-07-19T10:24:19.283Z
Answers.LastEditDate: 2024-07-19T10:24:19.283Z
Answers.UserDisplayName: Ian Ringrose
Answers.Body: <p>It <em>can</em> be done, and in fact it has already been done. Several times, even.</p>
  <p>A great starting point is the Wikipedia page &quot;Comparison of top chess players throughout history&quot;, in particular the paragraph <a href="https://en.wikipedia.org/wiki/Comparison_of_top_chess_players_throughout_history#Moves_played_compared_with_computer_choices" rel="nofollow noreferrer">Moves played compared with computer choices</a>.</p>
  <p>Let me quote a sentence from that Wiki article, about the Markovian model proposed by Jean-Marc Alliot of the Toulouse Computer Science Research Institute in 2017:</p>
  <blockquote>
  <p>These predictions have proven not only to be extremely close to the actual results when players have played concrete games against one another, but to also fare better than those based on Elo scores.</p>
  </blockquote>
  <p>Granted, this is applied only to the best players ever, to reduce the number of games that must be analysed. But I don't think applying this method to &quot;normal&quot; players is that impractical. Moves are already recorded, they just have to be saved to a PGN file and then the analysis is easy.</p>
  <p>My personal experience with Chess.com is that the Elo estimator (which, if I understand correctly, is based on a player's accuracy) works very poorly when evaluating bots that are rated 1000-1500 (e.g. a bot rated 1300 can be evaluated around 400). And I use bots exactly because in theory their gameplay should be consistent. Maybe the problem is that at lower levels there is too much variability for the accuracy to be a good indicator of a player's strength? This is something that some research could assess.</p>
  <p>So the approach does have some problems (like every other method, I'd say), but it also has some merits. Notably, it makes it possible to compare players who have lived at different times and who have never played against each other.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-17T16:39:44.587Z
Answers.LastActivityDate: 2024-07-17T16:39:44.587Z
Answers.UserDisplayName: Fabio says Reinstate Monica
Answers.Body: <p>The ELO system is bias towards a game's result (winning, drawing, losing). By this, I mean it doesn't care about what happens <em>in</em> a game, only the result <em>of</em> a game.</p>
  <p>From my understanding (these algorithms are always in flux) and simplifying, a computer rates a game based on the amount of centipawns advantage given up throughout the game (not a linear sum relationship). This is biased towards what happens <em>in</em> a game.</p>
  <p>Let's think about a player like Richard Rapport. He is famous for his crazy openings. He does something crazy, throwing their opponent out of prep. He then plays a killer mid and endgame.</p>
  <p>In your suggested system, Richard Rapport compared to other (current) top players would see his rating fall badly because he consistency gives up advantage in the early game. Whereas some players who play pretty bland games and tie mostly would see very large (relative) gains.</p>
  <p>Does it make sense for Rapport to be lowered rated than a player (a player he could beat more often than he loses to), just because that player plays straight, serious chess? That is not a rhetorical question.</p>
  <p>I am not saying one system is better than another but pointing out that they value two different things.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-17T16:47:23.693Z
Answers.Text: A "crazy" opening should lead to an inferior position (otherwise it wouldn't be crazy). If at that point Rapport can win, it means he is better at evaluating that unusual position and playing accurately. Summing everything up, a good metric should evaluate him better than his opponent. If that doesn't happen, the metric must be improved. So while I agree that the two systems value different things, I think they *can* lead to the same conclusion.
Answers.UserDisplayName: Fabio says Reinstate Monica
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-17T15:35:09.603Z
Answers.LastActivityDate: 2024-07-17T15:49:08.200Z
Answers.LastEditDate: 2024-07-17T15:49:08.200Z
Answers.UserDisplayName: Lan
Body: <p>A computer like Stockfish can evaluate the rating performance of individual players based on a game. Why don't we take the average rating evaluation of a player's games to measure the player's strength instead of having elo systems etc?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-07-16T17:07:29.043Z
Comments.Text: This is an interesting question, as such rating software may stop being usefull if chess players started to study how to get a higher rating from the software.
Comments.UserDisplayName: Ian Ringrose
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-07-17T08:36:52.860Z
Comments.Text: @IanRingrose "Goodhart's Law is expressed simply as: “When a measure becomes a target, it ceases to be a good measure.”"
Comments.UserDisplayName: StuperUser
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-07-17T15:45:55.043Z
Comments.Text: So if players were playing under duress constantly because they are in a warzone, those players would automatically be downgraded by your Stockfish vs players playing in stable countries in peace.
Comments.UserDisplayName: stackoverblown
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-07-18T14:24:51.347Z
Comments.Text: @stackoverblown:  I guess you are suggesting that a benefit of ELO is that it accounts for externalities such as the conditions (warzone) of a player's environment.  However, I thought this was usually considered a FAULT of ELO in that the rating of isolated groups are not in line with the rating of other groups.  For example, see this question https://chess.stackexchange.com/q/39132/32596
Comments.UserDisplayName: James
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-07-18T16:23:06.520Z
Comments.Text: It's funny Ian Ringrose's answer says this: > If the aim is to find the best player of all time using a computer systems that looks at the moves a player makes, it is clearly not possible as explained by other answers. because that exact premise has been done at least a few times! [Computer Analysis of World Chess Championship Players by Romero et al. (2019)](https://www.thinkmind.org/download.php?articleid=icsea_2019_9_40_10137) compared top player moves against Stockfish 10 to see which player was the most accurate. Not surprisingly, Carlsen comes out on top. A similar study was done in 2006
Comments.UserDisplayName: qwr
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-07-18T16:25:20.207Z
Comments.Text: @IanRingrose computers are so good at playing, that any player who gets a good rating for a computer would be undoubtedly a strong player against humans.
Comments.UserDisplayName: qwr
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-07-18T16:29:19.517Z
Comments.Text: @SecretAgentMan many of the answers here completely ignore the already published studies that do exactly this. The reality of statistical practice is that we can also quantify uncertainty and our caveats while still being scientific, without only appealing to theoretical arguments.
Comments.UserDisplayName: qwr
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-07-19T01:38:24.540Z
Comments.Text: Perhaps interesting to note that what the OP describes is exactly what they do in backgammon. They don't really have an ELO system (except on online sites) but there are master and grandmaster titles which are awarded based on a player's PR, which is how well a player plays compared to the world's best computer program. The reason they do this is (obviously) because the luck factor of the dice introduces a lot of variance into the results of individual matches.
Comments.UserDisplayName: Flounderer
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-07-23T19:20:47.757Z
Comments.Text: The only thing that matters is winning or drawing (without breaking rules). It shouldn't matter how objectively good your moves are. Maybe someone wants to play very aggressive but dubious moves because they know their opponents buckle under the pressure.
Comments.UserDisplayName: Leonard Blackburn
Tags: <rating><computer-chess><elo>
Title: Why not use computers to evaluate strength of players?
ThreadText: # Why not use computers to evaluate strength of players?
  
  (Asked by: username_0 on 2024-07-16)
  
  A computer like Stockfish can evaluate the rating performance of individual players based on a game. Why don't we take the average rating evaluation of a player's games to measure the player's strength instead of having elo systems etc?
  
  **Tags:** rating, computer-chess, elo
  
  > **Comments:**
  > **username_1**: This is an interesting question, as such rating software may stop being usefull if chess players started to study how to get a higher rating from the software.  
  > **username_2**: @IanRingrose "Goodhart's Law is expressed simply as: “When a measure becomes a target, it ceases to be a good measure.”"  
  > **username_3**: So if players were playing under duress constantly because they are in a warzone, those players would automatically be downgraded by your Stockfish vs players playing in stable countries in peace.  
  > **username_4**: @username_3:  I guess you are suggesting that a benefit of ELO is that it accounts for externalities such as the conditions (warzone) of a player's environment.  However, I thought this was usually considered a FAULT of ELO in that the rating of isolated groups are not in line with the rating of other groups.  For example, see this question https://chess.stackexchange.com/q/39132/32596  
  > **username_5**: It's funny username_1's answer says this: > If the aim is to find the best player of all time using a computer systems that looks at the moves a player makes, it is clearly not possible as explained by other answers. because that exact premise has been done at least a few times! [Computer Analysis of World Chess Championship Players by Romero et al. (2019)](https://www.thinkmind.org/download.php?articleid=icsea_2019_9_40_10137) compared top player moves against Stockfish 10 to see which player was the most accurate. Not surprisingly, Carlsen comes out on top. A similar study was done in 2006  
  > **username_5**: @IanRingrose computers are so good at playing, that any player who gets a good rating for a computer would be undoubtedly a strong player against humans.  
  > **username_5**: @SecretAgentMan many of the answers here completely ignore the already published studies that do exactly this. The reality of statistical practice is that we can also quantify uncertainty and our caveats while still being scientific, without only appealing to theoretical arguments.  
  > **username_6**: Perhaps interesting to note that what the OP describes is exactly what they do in backgammon. They don't really have an ELO system (except on online sites) but there are master and grandmaster titles which are awarded based on a player's PR, which is how well a player plays compared to the world's best computer program. The reason they do this is (obviously) because the luck factor of the dice introduces a lot of variance into the results of individual matches.  
  > **username_7**: The only thing that matters is winning or drawing (without breaking rules). It shouldn't matter how objectively good your moves are. Maybe someone wants to play very aggressive but dubious moves because they know their opponents buckle under the pressure.  
  
  ---
  
  ## 7 answers
  
  ### Answer by username_8 on 2024-07-16. Score: 19
  
  Elo is a system to summarize past results into a meaningful number that informs about the (relative) strength of a player. Since the only objectives a player has during a game of chess are getting a win or a draw, only success and failure in those two specific objectives should be taken into account. It's true that how accurately someone played in a game can be used to predict how many wins he'll get in the future, but at least if we want our rankings to have an "official" value (granting titles, qualifying for tournaments...), we should base those rankings on past results alone rather than trying to make predictions about potential future outcomes.
  
  But even if we want to use an engine-similarity ranking as just a way to determine who is the best player of all time (parallel to the official Elo rankings), it still comes with its share of problems:
  
  * First we'd need to choose a specific engine to use. If we just take the one with the highest Elo at the moment, we're going back to taking past results as the ultimate metric of strength. We'd also need to decide of how much time per move and on what hardware the engine should run. These are again arbitrary choices that could leave to dramatically different outcomes.
  * Since new and stronger engines are coming up all the time, we'd need to be constantly recalculating past ranking lists. It'd seem unfair to dismiss "Player A" as the strongest player of 2024 just because the engines of the time weren't good enough to understand the mistakes of their opponents.
  * Chess has many different time controls yet FIDE does fine with just 3 different ratings (Blitz, Rapid and Classical). An accuracy-based system would require to introduce some correction for specific time controls (or to generate separate lists for 60+30, 90+30 and so on).
  * Chess is a two-player game. Forcing your opponent into making a mistake is just as important as avoiding mistakes yourself. Engines have no concept of the "difficulty" of a move. For instance in a theoretically drawn endgame like rook+bishop vs rook, giving the bishop away for free is just as "accurate" as making a challenging move that forces your opponent into a difficult defense.
  * Quiet games usually lead to easier positions where fewer mistakes are played. An accuracy-based ranking would be heavily biased against aggressive players. Many of Tal's sacrifices are "blunders" even though they actually led to wins.
  * In an Elo system, you win the biggest amount of points by playing higher-rated opposition. If you just "farm" lower-rated players, your rating will increase very slowly. But it's much easier to get high accuracies against bad players, so you can massively inflate your score by brilliantly defeating newcomers.
  
  In short, a system that rewards a player for a Berlin draw against a beginner but punishes him for a win in a chaotic double-edge position against a Grandmaster can never be a valid way to measure chess strength.
  
  > **Comments:**
  > **username_9**: 'Engines have no concept of the "difficulty" of a move.' - I mean, some Engines definitely do have this concept.  They aren't pure search-to-solution systems.  Even against an equally smart AI, knowing that deep down a search tree there are moves/situations that are easier or harder for the other side to analyze has value for the AI.  
  > **username_8**: @username_9 yeah but that only really applies to engine vs engine play, not really at "correcting" accuracy scores to reflect human difficulty. Even if the computer could perfectly measure how difficult a position will be for a human, the compensation for game accuracy that it'd introduce would always be arbitrary.  
  > **username_10**: A lot of this does seem to be based on prejudices/lacking knowledge of technical possibilities. It seems well possible to overcome some of the criticism by designing a chess engine with the goal of measuring strength instead of playing to win. Other points (who defines what is best) are very valid though.  
  > **username_5**: Larry Kaufman addresses almost all of these points in his study https://www.chess.com/article/view/chess-accuracy-ratings-goat  
  
  ---
  
  ### Answer by username_11 on 2024-07-16. Score: 8
  
  > Why don't we take the average rating evaluation of a player's game to measure the player's strength instead of having elo systems etc?
  
  The answer is simple and obvious. The elo system requires only the results of games and the statistical calculations are straightforward and quick. What you suggest would require all the moves of all the over-the-board FIDE rated games be available before it could even start. Then enormous computer power would be required to analyse the hundreds of thousands of games played every month.
  
  That just applies to standard time control games. It would be completely impossible to do for the vast majority of blitz and rapid games which are not played on electronic DGT boards which record the moves. There is no record of the moves played hence rapid and blitz ratings using this system would be impossible.
  
  
  ---
  
  ### Answer by username_12 on 2024-07-16. Score: 6
  
  Complementary to the other more pragmatic answers let's take a step back and approach this from a more theoretical angle: The essence of a rating system is the prediction of the result of a game between two players. And since human players typically do not play deterministically, they essentially constitute or employ a mixed strategy from a game theoretical perspective. Therefore, the rating is a predictor for the statistically expected score between two such mixed strategies with only limited information about these strategies/players.
  
  Therefore you need two things to define a rating:
  
  1. Knowledge about the reference population (rating pool) of strategies.
  2. Knowledge about the strategy/player to be rated.
  
  What is the optimal way to use this knowledge to construct a predictor for the expected score? To me, it is not a given that only using game results is the optimal way. If you could build an approximate model of the mixed strategy and the reference strategies (e.g., using machine learning) from game records, you could do a monte-carlo simulation to give an expected score and hence a rating.
  
  All I want to say with this is that while it might practically currently not be very accurate or feasible to predict playing strength from just moves without game results, it is far from an absurd question. It is just that purely result based rating estimation has been the most transparent and robust so far, especially given the incomplete knowledge and constant changes of both the player pool and individual players.
  
  > **Comments:**
  > **username_12**: Every rating system in some form or another works by comparing the achieved result against the expected result to be able to update rating. If there is no way to calculate an expected result, then how do you update rating other than recalculating from scratch? And a similar rating adjustment scenario to what you described happened at the beginning of this year since Elo got worse at predicting results, so FIDE seems to disagree with you. https://www.fide.com/news/2831  
  > **username_13**: @username_8: Arpad Elo very explicitly constructed the Elo rating system around the idea that a player's skill level should be predictive of how many games they win and lose against other players (of varying skill levels). If a player's Elo rating does not correctly reflect the true probability of winning, they should gain or lose points until this is no longer the case (but that process might take many games).  
  > **username_12**: Yes, the only thing that has largely remained the same the whole time is the calculation of the expected score, because that is the foundation. The update system keeps being changed just so that ratings reflect the expected scores again, i.e., are accurate/stable. But this discussion has become too long, anyone still in doubt can form his own opinion just reading from FIDE's own rating system consultant https://www.fide.com/docs/presentations/Sonas%20Proposal%20-%20Repairing%20the%20FIDE%20Standard%20Elo%20Rating%20System.pdf  
  
  ---
  
  ### Answer by username_14 on 2024-07-16. Score: 6
  
  Stockfish does not evaluate the rating performance of a player in a game, it can only give an opinion on the accuracy of a player's moves. I say "opinion" because stockfish does (rarely) lose games to other computers, so it is not 100% accurate.  
  At master level, certain openings can be played with high accuracy very easily. Other openings are too complicated for a human to navigate with computer precision. This does not matter to the player, because they are playing other humans. Should a player receive a higher rating because they are consistently drawing with 98% accuracy in simple openings rather than seizing advantages in messy, imperfect middlegames? I don't think so.  
  Some sites like chess.com offer an estimate of a player's Elo following every game. I think this is mostly a novelty and another way to "gamify" chess improvement. The numbers it gives should be taken with a grain of salt.
  
  EDIT: username_11 answer is maybe the "real" answer here. Even if a perfect engine was created that could also judge how difficult moves are to find from a human perspective, tournament directors don't want to have to input all moves from every game. It may seem like that's something they already do since you can find massive databases of master games. However, it's easy to forget that FIDE and USCF sanction tournaments for all skill levels. The Elo system is fast and ***practical***, it allows a player to receive an official rating using only their tournament results.
  
  > **Comments:**
  > **username_9**: I mean, Stockfish also provides board ratings (the black/white bar) - if you measure the impact of each player's move on that board rating, you get "more" than accuracy, in that a non-perfect move that results in nearly as good board position is different than a non-perfect move that results in a bad board position.  "The average move was -0.5 off ideal against a foe whose average move was -1.0 off ideal" could be turned into a reasonable rating system; but even here, we end up factoring in how good the opponent is, as "force the opponent to play badly" is a chess skill.  
  > **username_14**: @username_9 it sounds like you're describing accuracy. At amateur level that is maybe a good metric for strength. Once you reach strong master level, I don't think accuracy is a good way to rank strength.  
  
  ---
  
  ### Answer by username_1 on 2024-07-17. Score: 4
  
  If the aim is to ***find the best player of all time*** using a computer systems that looks at the moves a player makes, it is clearly not possible as explained by other answers.
  
  If we believe
  
  > The essence of a rating system is the prediction of the result of a
  > game between two players
  
  We can't hope to use a computer based measurement of the accuracy of a player's moves to replace rating systems, I will not repeat other answers that expain why.
  
  What what if we believe?
  
  > The primary aim of a rating system to to divide players in a chess
  > festival between separate Swiss Tournaments so it is likely that most
  > games will be fun, eg the outcome is not predictable.
  
  And we also believe?
  
  > Players who have not played in formal tournaments against anyone
  > outside of their school chess club should be able to be allocated to
  > the correct Switt Tournament so they get many fun games
  >
  > This includes when none of the club members have regularly played
  > people with a reliable ratings
  
  Then we have an interesting and solvable problem with two likely solutions:
  
  * Use a computer to get accuracy of a player's moves, then use the result as the 1st approximation of their rating.
  * Get them to play agaist a system like [Maia](https://maiachess.com/#:%7E:text=Maia%20is%20an%20AlphaZero%2FLeela,milestone%20between%201100%20and%201900.), finding the level that they tend to win as many games as lose, then use the result as the 1st approximation of their rating.
  
  Once we have that 1st approximation of their rating and the 1st approximation of the rating ratings of other members of the schools chess club we can combine them with known recent game results from the club and feed the information into a linear programming solver to give an improved approximation of their ratings.
  
  In other words we could consider each school chess club to be a disconnected *rating pool*, and use the computer based methods to work out an approximation of the mapping function between these rating pools.
  
  ---
  
  But we could likely practically do as well by using the opponents Chess.com rating to work out the "tournament rating" of each person in a rating pools who played recent OTB games, then use these "tournament ratings" to work out the mapping between each chess clubs internal rating system. But that is no what the question asked!
  
  > **Comments:**
  > **username_5**: Regarding your first sentence: there was a paper that did exactly that to determine the most accurate player of all time.  
  > **username_1**: @username_5 The best players study their opponents and play differently defending on the opponents, so past play can't predict how they will play someone they never played before but greatly respects.  
  > **username_5**: Playing style has already been incorporated into studies on historical players. In fact, it is very possible to predict play *and* validate our estimates statistically.  
  
  ---
  
  ### Answer by username_15 on 2024-07-17. Score: 2
  
  It *can* be done, and in fact it has already been done. Several times, even.
  
  A great starting point is the Wikipedia page "Comparison of top chess players throughout history", in particular the paragraph [Moves played compared with computer choices](https://en.wikipedia.org/wiki/Comparison_of_top_chess_players_throughout_history#Moves_played_compared_with_computer_choices).
  
  Let me quote a sentence from that Wiki article, about the Markovian model proposed by Jean-Marc Alliot of the Toulouse Computer Science Research Institute in 2017:
  
  > These predictions have proven not only to be extremely close to the actual results when players have played concrete games against one another, but to also fare better than those based on Elo scores.
  
  Granted, this is applied only to the best players ever, to reduce the number of games that must be analysed. But I don't think applying this method to "normal" players is that impractical. Moves are already recorded, they just have to be saved to a PGN file and then the analysis is easy.
  
  My personal experience with Chess.com is that the Elo estimator (which, if I understand correctly, is based on a player's accuracy) works very poorly when evaluating bots that are rated 1000-1500 (e.g. a bot rated 1300 can be evaluated around 400). And I use bots exactly because in theory their gameplay should be consistent. Maybe the problem is that at lower levels there is too much variability for the accuracy to be a good indicator of a player's strength? This is something that some research could assess.
  
  So the approach does have some problems (like every other method, I'd say), but it also has some merits. Notably, it makes it possible to compare players who have lived at different times and who have never played against each other.
  
  
  ---
  
  ### Answer by username_16 on 2024-07-17. Score: 2
  
  The ELO system is bias towards a game's result (winning, drawing, losing). By this, I mean it doesn't care about what happens *in* a game, only the result *of* a game.
  
  From my understanding (these algorithms are always in flux) and simplifying, a computer rates a game based on the amount of centipawns advantage given up throughout the game (not a linear sum relationship). This is biased towards what happens *in* a game.
  
  Let's think about a player like Richard Rapport. He is famous for his crazy openings. He does something crazy, throwing their opponent out of prep. He then plays a killer mid and endgame.
  
  In your suggested system, Richard Rapport compared to other (current) top players would see his rating fall badly because he consistency gives up advantage in the early game. Whereas some players who play pretty bland games and tie mostly would see very large (relative) gains.
  
  Does it make sense for Rapport to be lowered rated than a player (a player he could beat more often than he loses to), just because that player plays straight, serious chess? That is not a rhetorical question.
  
  I am not saying one system is better than another but pointing out that they value two different things.
  
  > **Comments:**
  > **username_15**: A "crazy" opening should lead to an inferior position (otherwise it wouldn't be crazy). If at that point Rapport can win, it means he is better at evaluating that unusual position and playing accurately. Summing everything up, a good metric should evaluate him better than his opponent. If that doesn't happen, the metric must be improved. So while I agree that the two systems value different things, I think they *can* lead to the same conclusion.  
  
  ---
  
  