Answers.Body: <p>Computer chess engine ratings are <strong>not</strong> related to the FIDE rating list.</p>
  
  <p>For example, the <code>MicroMax</code> engine has around 1950 rating on CCRL:</p>
  
  <blockquote>
    <p><a href="http://www.computerchess.org.uk/ccrl/4040/" rel="noreferrer">http://www.computerchess.org.uk/ccrl/4040/</a></p>
  </blockquote>
  
  <p>I distribute an iOS app (<a href="https://itunes.apple.com/us/app/chess-mini/id616424231?mt=8" rel="noreferrer">ChessMini</a>) for this engine. I wrote on the app description page: </p>
  
  <blockquote>
    <p>The engine has a rating of Elo 2000 on the Computer Chess Rating Lists and is stronger than 70% of the human chess players.</p>
  </blockquote>
  
  <p>This is true only if the CCRL chess rating is a close approximation to the FIDE rating list.</p>
  
  <p>Unfortunately, I've received several users complained that they were able to beat the app convincingly despite their FIDE ratings are much less than 1900. In fact, I have a FIDE rating around 2000 myself but the MicroMax engine has never beaten me. I think it's "real" rating on the FIDE scale is 1300-1400.</p>
  
  <p><strong>Forget about the human rating list, just look at the relative ranking</strong>.</p>
  
  <p>Computer chess ratings are <strong>extraordinary accurate</strong> (but no relationship to FIDE) because the sample size is only bounded by your processing power. You start an engine tournament, leave the computer running and goto bed... It's 24 hours chess like TCEC. You play as many engine matches as your computer can afford.</p>
  
  <p>Computer engines get a rating by playing other engines. No human intervention. No grandmaster. Anybody with a laptop can do that:</p>
  
  <ul>
  <li>Download <a href="http://www.playwitharena.com/" rel="noreferrer">Arena</a> or <a href="https://github.com/cutechess/cutechess" rel="noreferrer">Cutechess</a></li>
  <li>Download your favorite engines</li>
  <li>Start an engine tournament</li>
  </ul>
  
  <p>That's it, it's so simple! You'll get estimated ratings for your engines.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-10-20T13:53:49.193Z
Answers.Text: Still there should be some approximate relation between the two lists no? (It's hard to find such a relation with few games played between humans and computers, but there should be one.)
Answers.UserDisplayName: TMM
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-10-20T13:54:40.150Z
Answers.Text: @TMM Maybe. If there is one and it can be proven statistically (something like correlation), please attempt your answer.
Answers.UserDisplayName: SmallChess
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-05-19T12:16:18.983Z
Answers.Text: Your answer offers an interesting anecdote and is valuable to that extent, but does not seem authoritative in the specific matter OP raises. Your answer is your answer, so I do not think that you should change it unless you just want to; but I nevertheless believe that the answer would have been more accurate had it begun, "I do not know the answer to your question, but here is an illuminating anecdote." In my view, therefore, the question is *still open* and still wants a proper answer.
Answers.UserDisplayName: thb
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-10-20T02:23:13.580Z
Answers.LastActivityDate: 2017-10-20T02:36:54.093Z
Answers.LastEditDate: 2017-10-20T02:36:54.093Z
Answers.UserDisplayName: SmallChess
Body: <p>In light of the ongoing TCEC tournament, I was thinking, how do they come up with the ratings of these computers? 
  Presumably, a 2000 rated chess engine should expect a score of about .5 against a human rated 2000. This perhaps makes it easy to verify if chess engines are really approximately the rating they say they are. However, this is problematic for higher rated programs like komodo, who is supposedly around 3250. Obviously, no humans play at this level. Furthermore, even 2800 rated computers do not play against humans(I doubt magnus plays against computers to help developers define their rating). It is even suspicious if engines play against grandmasters to determine their strength. So, I suspect that these engine ratings are determined by engines playing opponents of similar strength a long time ago, and most ratings come from engine vs engine games these days. But if this was true, I wouldn't expect that engine ratings are accurate in comparison to the human elo scale. Is there a way that they check that engine ratings are accurate? </p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-10-28T06:08:37.023Z
Comments.Text: If you can get a reliable rated human, play a match vs a weak computer, to "calibrate" the computer elo, then later you may use that engine vs other engine 300 points higher to calibrate that one, and go staircase like that, 300 elo points each time to make it statistically easy (so that not all are wins, because then how do you know how much points better it is) i think it would be feasible to at least esstimate elo of strong engines.
Comments.UserDisplayName: Santropedro
Tags: <engines>
Title: How accurate are chess engine ratings?
ThreadText: # How accurate are chess engine ratings?
  
  (Asked by: username_0 on 2017-10-20)
  
  In light of the ongoing TCEC tournament, I was thinking, how do they come up with the ratings of these computers?
  Presumably, a 2000 rated chess engine should expect a score of about .5 against a human rated 2000. This perhaps makes it easy to verify if chess engines are really approximately the rating they say they are. However, this is problematic for higher rated programs like komodo, who is supposedly around 3250. Obviously, no humans play at this level. Furthermore, even 2800 rated computers do not play against humans(I doubt magnus plays against computers to help developers define their rating). It is even suspicious if engines play against grandmasters to determine their strength. So, I suspect that these engine ratings are determined by engines playing opponents of similar strength a long time ago, and most ratings come from engine vs engine games these days. But if this was true, I wouldn't expect that engine ratings are accurate in comparison to the human elo scale. Is there a way that they check that engine ratings are accurate?
  
  **Tags:** engines
  
  > **Comments:**
  > **username_1**: If you can get a reliable rated human, play a match vs a weak computer, to "calibrate" the computer elo, then later you may use that engine vs other engine 300 points higher to calibrate that one, and go staircase like that, 300 elo points each time to make it statistically easy (so that not all are wins, because then how do you know how much points better it is) i think it would be feasible to at least esstimate elo of strong engines.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2017-10-20. Score: 6
  
  Computer chess engine ratings are **not** related to the FIDE rating list.
  
  For example, the `MicroMax` engine has around 1950 rating on CCRL:
  
  > <http://www.computerchess.org.uk/ccrl/4040/>
  
  I distribute an iOS app ([ChessMini](https://itunes.apple.com/us/app/chess-mini/id616424231?mt=8)) for this engine. I wrote on the app description page:
  
  > The engine has a rating of Elo 2000 on the Computer Chess Rating Lists and is stronger than 70% of the human chess players.
  
  This is true only if the CCRL chess rating is a close approximation to the FIDE rating list.
  
  Unfortunately, I've received several users complained that they were able to beat the app convincingly despite their FIDE ratings are much less than 1900. In fact, I have a FIDE rating around 2000 myself but the MicroMax engine has never beaten me. I think it's "real" rating on the FIDE scale is 1300-1400.
  
  **Forget about the human rating list, just look at the relative ranking**.
  
  Computer chess ratings are **extraordinary accurate** (but no relationship to FIDE) because the sample size is only bounded by your processing power. You start an engine tournament, leave the computer running and goto bed... It's 24 hours chess like TCEC. You play as many engine matches as your computer can afford.
  
  Computer engines get a rating by playing other engines. No human intervention. No grandmaster. Anybody with a laptop can do that:
  
  * Download [Arena](http://www.playwitharena.com/) or [Cutechess](https://github.com/cutechess/cutechess)
  * Download your favorite engines
  * Start an engine tournament
  
  That's it, it's so simple! You'll get estimated ratings for your engines.
  
  > **Comments:**
  > **username_3**: Still there should be some approximate relation between the two lists no? (It's hard to find such a relation with few games played between humans and computers, but there should be one.)  
  > **username_2**: @username_3 Maybe. If there is one and it can be proven statistically (something like correlation), please attempt your answer.  
  > **username_4**: Your answer offers an interesting anecdote and is valuable to that extent, but does not seem authoritative in the specific matter OP raises. Your answer is your answer, so I do not think that you should change it unless you just want to; but I nevertheless believe that the answer would have been more accurate had it begun, "I do not know the answer to your question, but here is an illuminating anecdote." In my view, therefore, the question is *still open* and still wants a proper answer.  
  
  ---
  
  