Answers.Body: <p>For a very accurate rank of a player's quality, you can use the excellent tool provided by www.chess-db.com. It lets you upload your games and after some minutes it outputs the quality of both players in percentage compared with the best moves of a strong engine.</p>
  
  <p>This is the page to upload a PGN file:
  <a href="http://chess-db.com/public/game_upload.jsp" rel="noreferrer">http://chess-db.com/public/game_upload.jsp</a></p>
  
  <p>And this is an example of the results:
  <a href="http://chess-db.com/public/game.jsp?id=Pablo%20Bento.Shredder%20Android.107755008" rel="noreferrer">http://chess-db.com/public/game.jsp?id=Pablo%20Bento.Shredder%20Android.107755008</a></p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2013-12-09T17:44:00.660Z
Answers.LastActivityDate: 2013-12-09T17:44:00.660Z
Answers.UserDisplayName: user1657
Answers.Body: <p>chess.com CAPS. Compare CAPS score from chess.com (requires subscription), to table found in a graphic on this: 
  <a href="https://www.chess.com/article/view/better-than-ratings-chess-com-s-new-caps-system" rel="nofollow noreferrer">https://www.chess.com/article/view/better-than-ratings-chess-com-s-new-caps-system</a> .
  Also of interest: 
  <a href="https://www.chess.com/article/view/who-was-the-best-world-chess-champion-in-history" rel="nofollow noreferrer">https://www.chess.com/article/view/who-was-the-best-world-chess-champion-in-history</a></p>
  
  <p>Note of caution: Caps scores for anyone particular game are volatile; best to make some average of CAPS scores over a batch of games. Right now, to my knowledge, CAPS can, by extension, predict an Elo rating from a set of moves.
  Another note: CAPS scores exist in a vacuum without respect to the time controls. I play better chess at slower time controls than I do in a 1-minute bullet game. CAPS will see this difference in play strength and accordingly assign a lower rating to the bullet games. This does not mean that I am not the same person who played the slower time control games!</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-09-28T13:17:53.793Z
Answers.LastActivityDate: 2018-09-28T13:17:53.793Z
Answers.UserDisplayName: JWILD
Answers.Body: <p>What you're asking does not exist by my knowledge. However, this is my idea:</p>
  
  <p>You'll need samples with the following features, (1) Chess position, (2) Move made in the position, (3) Rating of the player that made the move.</p>
  
  <p>Let's say you have 1 billion samples. You can train a computer algorithm on these samples that can predict for each move in a position the quality of the move with respect to a rating. All ratings for all moves can be averaged out to get the approximate rating of the player and in effect the quality of his game.</p>
  
  <p>This is a rough idea that can be further polished.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-09-29T04:42:06.813Z
Answers.Text: This doesn't really work. In most games, you will relatively quickly be in positions not in the database - yes, even if you somehow get access to a billion game database.  And you get punished if your opponent plays weakly - after 1.e4 e5 2.Qh5, no matter what Black plays it's going to have a low average rating, because that opening is played way more at lower levels than higher ones.  (Heck, even 1...e5 itself probably has a rather low average rating.) What happens *after* the known openings is way more indicative of a player's strength than how long they follow theory.
Answers.UserDisplayName: D M
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-10-01T05:08:13.823Z
Answers.Text: I gave this answer 5 years before AlphaZero. Now we can all just ask AlphaZero to annotate our games for us and tell us how well we are doing. We are probably almost at the point that we can even ask WHY a move is good or bad. That's what you really need to improve. Current human ratings are overestimated.
Answers.UserDisplayName: Rafiek
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2013-11-22T22:17:59.467Z
Answers.LastActivityDate: 2013-11-22T22:17:59.467Z
Answers.UserDisplayName: Rafiek
Answers.Body: <p>The Site ratings at <em>slow time controls</em> <strong>can</strong> be quite reliable for servers where strong players congregate (ICC, FICS to name a few) as the ratings VERY closely reflect your true playing strength if you've played enough games. For very standardized rating systems such as USCF and FIDE/ELO, you will notice that the different rating classes tend to point to the <strong>types of mistakes</strong> those players are still making. NM Dan Heisman's <a href="http://rads.stackoverflow.com/amzn/click/0979148243" rel="noreferrer">Improving Chess Thinker</a> does an excellent job discussing the types of errors players make across the rating classes.</p>
  
  <p>Have you tried the many self-test books out there?  Igor Khmelnitsky's <a href="http://www.chessexam.com/" rel="noreferrer">Chess Rating Exam</a> and  Danny Kopec's <a href="http://rads.stackoverflow.com/amzn/click/0970085214" rel="noreferrer">Test, Evaluate and Improve your chess</a> are excellent books that allow you to track your progress by seeing how you perform against graded test positions.</p>
  
  <p>Your compare-my-moves-with-an-engine approach is another way to do this but once again, the ??/? moves are really only indicating tactical errors, not strategic or positional or even behavioral or time-management mistakes you might be making.</p>
  
  <p>That's why playing slow time-control OTB/online games against equal-to-stronger opposition and getting them reviewed + critiqued by stronger players is an efficient way to improve. Your mistakes in every category (tactics, knowledge, thought process, time management etc.) get highlighted and you can simply measure progress in <strong>terms of the mistakes you've stopped making.</strong></p>
  
  <p>Though one fun variant you can try with an engine at home: Why not extend your engine-evaluation method to visually observe a player's quality/performance via evaluation graphs?  In other words, take engine evaluation scores <strong>per move</strong> and plot them (some free software like SCID does this for you) over the moves.</p>
  
  <p>For example: Two rank beginners would have a game that looks like:</p>
  
  <p><img src="https://i.sstatic.net/ix4jl.png" alt="enter image description here"></p>
  
  <p>Notice how jagged these are. Both sides make many terrible mistakes (slopes of the spikes!) and <strong>also</strong> how often they fail to exploit the other person's terrible mistakes.</p>
  
  <p>The spikes are always fun to look at : 
  <img src="https://i.sstatic.net/0Si7y.png" alt="enter image description here"></p>
  
  <p>Two intermediate (USCF 1400-1600) players might have games that look like:
  <img src="https://i.sstatic.net/LbP6P.png" alt="enter image description here"></p>
  
  <p>It does look jagged, but notice how the y-axis (engine evaluation) is way smaller ... indicating that these players are more seasoned and play higher quality chess than the novices.</p>
  
  <p>For a final comparison, a 1911 Grandmaster game would look like this:</p>
  
  <p><img src="https://i.sstatic.net/2sD2n.png" alt="enter image description here"></p>
  
  <p>No comments necessary here :)  These guys really don't make many mistakes, do they?</p>
  
  <p>If you could devise your own heuristic for mapping the slopes + scale of an evaluation graph to  <strong>player skill/performance</strong>, perhaps this is one way to go? :)</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-11-28T10:59:10.947Z
Answers.Text: What program do you use to generate these graph? I've scid + stockfish, do I have this option?
Answers.UserDisplayName: Uri London
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2013-11-24T12:06:19.033Z
Answers.Text: Note that I'd like to throw in a caveat that Engine evaluation scores are not so reliable in some nuanced positions and some material-hungry flavors will consider a Gambit type-opening very differently than a human would. How long you set your engine on a half-move position while going over the game will also influence things a bit. Be warned! :)
Answers.UserDisplayName: shivsky
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2013-11-22T21:35:18.457Z
Answers.LastActivityDate: 2013-11-22T21:52:58.623Z
Answers.LastEditDate: 2013-11-22T21:52:58.623Z
Answers.UserDisplayName: shivsky
Body: <p>Suppose I have a completed game. I don't have Elo ratings of the players. My purpose is to evaluate a player's performance in the game based solely on his moves. Can this can be done automatically using a chess program?</p>
  
  <p>The result can be his approximate Elo rating, or just some value indicating his strength or error rate.</p>
  
  <p>If it helps, a database of the player's games can be given. Again, with no Elo ratings.</p>
  
  <p>My motivation is simple. I play chess over the internet and would like to automatically track my progress, based on the games themselves, not on the rating on the sites. I'm a (upper)-beginner level.</p>
  
  <p>A simple solution is to annotate the game using any computer engine and track number of ?!, ? and ?? marks. However, it's not very accurate, and I'd like to get more ideas :)</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-09-28T16:07:09.387Z
Comments.Text: Any evaluation based on a single game will come with a huge error margin. Your "progress" will be going up and down constantly and I doubt that you will be able to track anything besides very long term "progress" by this method (which basically comes down to an average over games). Chess ratings (or ratings in just about any other sport)  avoid this problem and I don't see anything wrong with using the online rating as indicator for your strength.
Comments.UserDisplayName: user1583209
Tags: <analysis><rating><elo><evaluation><performance>
Title: How to automatically evaluate a player's performance in a game?
ThreadText: # How to automatically evaluate a player's performance in a game?
  
  (Asked by: username_0 on 2013-11-22)
  
  Suppose I have a completed game. I don't have Elo ratings of the players. My purpose is to evaluate a player's performance in the game based solely on his moves. Can this can be done automatically using a chess program?
  
  The result can be his approximate Elo rating, or just some value indicating his strength or error rate.
  
  If it helps, a database of the player's games can be given. Again, with no Elo ratings.
  
  My motivation is simple. I play chess over the internet and would like to automatically track my progress, based on the games themselves, not on the rating on the sites. I'm a (upper)-beginner level.
  
  A simple solution is to annotate the game using any computer engine and track number of ?!, ? and ?? marks. However, it's not very accurate, and I'd like to get more ideas :)
  
  **Tags:** analysis, rating, elo, evaluation, performance
  
  > **Comments:**
  > **username_1**: Any evaluation based on a single game will come with a huge error margin. Your "progress" will be going up and down constantly and I doubt that you will be able to track anything besides very long term "progress" by this method (which basically comes down to an average over games). Chess ratings (or ratings in just about any other sport)  avoid this problem and I don't see anything wrong with using the online rating as indicator for your strength.  
  
  ---
  
  ## 4 answers
  
  ### Answer by username_6 on 2013-11-22. Score: 17 | Accepted answer
  
  The Site ratings at *slow time controls* **can** be quite reliable for servers where strong players congregate (ICC, FICS to name a few) as the ratings VERY closely reflect your true playing strength if you've played enough games. For very standardized rating systems such as USCF and FIDE/ELO, you will notice that the different rating classes tend to point to the **types of mistakes** those players are still making. NM Dan Heisman's [Improving Chess Thinker](http://rads.stackoverflow.com/amzn/click/0979148243) does an excellent job discussing the types of errors players make across the rating classes.
  
  Have you tried the many self-test books out there? Igor Khmelnitsky's [Chess Rating Exam](http://www.chessexam.com/) and Danny Kopec's [Test, Evaluate and Improve your chess](http://rads.stackoverflow.com/amzn/click/0970085214) are excellent books that allow you to track your progress by seeing how you perform against graded test positions.
  
  Your compare-my-moves-with-an-engine approach is another way to do this but once again, the ??/? moves are really only indicating tactical errors, not strategic or positional or even behavioral or time-management mistakes you might be making.
  
  That's why playing slow time-control OTB/online games against equal-to-stronger opposition and getting them reviewed + critiqued by stronger players is an efficient way to improve. Your mistakes in every category (tactics, knowledge, thought process, time management etc.) get highlighted and you can simply measure progress in **terms of the mistakes you've stopped making.**
  
  Though one fun variant you can try with an engine at home: Why not extend your engine-evaluation method to visually observe a player's quality/performance via evaluation graphs? In other words, take engine evaluation scores **per move** and plot them (some free software like SCID does this for you) over the moves.
  
  For example: Two rank beginners would have a game that looks like:
  
  ![enter image description here](https://i.sstatic.net/ix4jl.png)
  
  Notice how jagged these are. Both sides make many terrible mistakes (slopes of the spikes!) and **also** how often they fail to exploit the other person's terrible mistakes.
  
  The spikes are always fun to look at :
  ![enter image description here](https://i.sstatic.net/0Si7y.png)
  
  Two intermediate (USCF 1400-1600) players might have games that look like:
  ![enter image description here](https://i.sstatic.net/LbP6P.png)
  
  It does look jagged, but notice how the y-axis (engine evaluation) is way smaller ... indicating that these players are more seasoned and play higher quality chess than the novices.
  
  For a final comparison, a 1911 Grandmaster game would look like this:
  
  ![enter image description here](https://i.sstatic.net/2sD2n.png)
  
  No comments necessary here :) These guys really don't make many mistakes, do they?
  
  If you could devise your own heuristic for mapping the slopes + scale of an evaluation graph to **player skill/performance**, perhaps this is one way to go? :)
  
  > **Comments:**
  > **username_6**: Note that I'd like to throw in a caveat that Engine evaluation scores are not so reliable in some nuanced positions and some material-hungry flavors will consider a Gambit type-opening very differently than a human would. How long you set your engine on a half-move position while going over the game will also influence things a bit. Be warned! :)  
  > **username_7**: What program do you use to generate these graph? I've scid + stockfish, do I have this option?  
  
  ---
  
  ### Answer by username_2 on 2013-12-09. Score: 5
  
  For a very accurate rank of a player's quality, you can use the excellent tool provided by www.chess-db.com. It lets you upload your games and after some minutes it outputs the quality of both players in percentage compared with the best moves of a strong engine.
  
  This is the page to upload a PGN file:
  <http://chess-db.com/public/game_upload.jsp>
  
  And this is an example of the results:
  <http://chess-db.com/public/game.jsp?id=Pablo%20Bento.Shredder%20Android.107755008>
  
  
  ---
  
  ### Answer by username_3 on 2018-09-28. Score: 2
  
  chess.com CAPS. Compare CAPS score from chess.com (requires subscription), to table found in a graphic on this:
  <https://www.chess.com/article/view/better-than-ratings-chess-com-s-new-caps-system> .
  Also of interest:
  <https://www.chess.com/article/view/who-was-the-best-world-chess-champion-in-history>
  
  Note of caution: Caps scores for anyone particular game are volatile; best to make some average of CAPS scores over a batch of games. Right now, to my knowledge, CAPS can, by extension, predict an Elo rating from a set of moves.
  Another note: CAPS scores exist in a vacuum without respect to the time controls. I play better chess at slower time controls than I do in a 1-minute bullet game. CAPS will see this difference in play strength and accordingly assign a lower rating to the bullet games. This does not mean that I am not the same person who played the slower time control games!
  
  
  ---
  
  ### Answer by username_4 on 2013-11-22. Score: 1
  
  What you're asking does not exist by my knowledge. However, this is my idea:
  
  You'll need samples with the following features, (1) Chess position, (2) Move made in the position, (3) Rating of the player that made the move.
  
  Let's say you have 1 billion samples. You can train a computer algorithm on these samples that can predict for each move in a position the quality of the move with respect to a rating. All ratings for all moves can be averaged out to get the approximate rating of the player and in effect the quality of his game.
  
  This is a rough idea that can be further polished.
  
  > **Comments:**
  > **username_5**: This doesn't really work. In most games, you will relatively quickly be in positions not in the database - yes, even if you somehow get access to a billion game database.  And you get punished if your opponent plays weakly - after 1.e4 e5 2.Qh5, no matter what Black plays it's going to have a low average rating, because that opening is played way more at lower levels than higher ones.  (Heck, even 1...e5 itself probably has a rather low average rating.) What happens *after* the known openings is way more indicative of a player's strength than how long they follow theory.  
  > **username_4**: I gave this answer 5 years before AlphaZero. Now we can all just ask AlphaZero to annotate our games for us and tell us how well we are doing. We are probably almost at the point that we can even ask WHY a move is good or bad. That's what you really need to improve. Current human ratings are overestimated.  
  
  ---
  
  