Elo sucks – better multiplayer rating systems for smaller games (2019)
Elo is great for what it was built for: ranking chess players. Chess is (1) extremely low-variance, (2) has an extremely high skill ceiling, and (3) is 1-on-1. Elo works great for chess, but it would <i>never</i> work for something like Poker. Let&#x27;s briefly go over these three points.<p>Most games aren&#x27;t chess -- where the only variance is picking who&#x27;s black and who&#x27;s white -- in fact, they might include dozens of RNG mechanics (from critical strikes to ability rolls, to spawn points). These mechanics (while fun and well-designed) might pollute your &quot;idealized&quot; model. There&#x27;s also the problem of RPS (rock-paper-scissors) mechanics or pick-counter-pick mechanics which will also heavily skew win rates. For instance, given a slow combo Magic deck, you will most likely auto-concede to mono red aggro (regardless of skill level). If you&#x27;re using Elo, this will pollute your model. (Hint: you shouldn&#x27;t be using Elo.)<p>Most games also don&#x27;t have chess&#x27; high skill ceiling. Chess has such a high skill ceiling for a number of reasons -- it&#x27;s one of the oldest games still being actively played, for one. Suppose your &quot;game&quot; is simply the flip of a coin (everyone wins 50% of the time). Zero skill involved. Trying to model win-loss-ratios using a sigmoid curve is silly. Obviously, no game is going to be a coin flip, but there&#x27;s a world of difference between chess and DOTA.<p>TruSkill attempts to fix (3) by using clever Bayesian updating on a player-by-player basis[1] but in reality, it&#x27;s a shit-show. Using Elo (or variants thereof) for team-based games where the team isn&#x27;t really a team (more like 3-5 random people plopped together for one match) is incredibly misguided, but continues to be implemented in just about every modern multiplayer game (to the players&#x27; frustration). Of course, mixing and matching pre-made groups with non pre-made groups creates as many issues as you might imagine.<p>In short, why so many game devs are enamored with Elo when it comes to ranking is a bit bizarre.<p>[1] <a href="https:&#x2F;&#x2F;www.microsoft.com&#x2F;en-us&#x2F;research&#x2F;wp-content&#x2F;uploads&#x2F;2007&#x2F;01&#x2F;NIPS2006_0688.pdf" rel="nofollow">https:&#x2F;&#x2F;www.microsoft.com&#x2F;en-us&#x2F;research&#x2F;wp-content&#x2F;uploads&#x2F;...</a>