Best Font for Online Reading: No Single Answer
Big warning: their summary is &quot;Among high-legibility fonts, a study found 35% difference in reading speeds between the best and the worst.&quot;<p>This is completely wrong and comes from an abuse of statistics. See the original research at <a href="https:&#x2F;&#x2F;dl.acm.org&#x2F;doi&#x2F;10.1145&#x2F;3502222#d1e6428" rel="nofollow">https:&#x2F;&#x2F;dl.acm.org&#x2F;doi&#x2F;10.1145&#x2F;3502222#d1e6428</a><p>An understandable explanation: imagine having 5 dice. You roll each die 4 times, then compare the highest sum to the lowest sum. Then you report that the highest die rolls 35% higher than the lowest. This is what the authors did, with each die being a font. But the experiment does not actually show evidence of any difference. If you rolled 500 dice, this method could claim that the highest dice are 200% higher than the lowest, even though all dice are still equal.<p>The original authors seem aware of this shortcoming, but did it anyway: &quot;we are somewhat stretching the applicability of a Cohen’s d analysis for this data&quot;. This is likely because they did not know of a better method. But it is wrong to be pushing this analysis. The main author is from industry, so perhaps he was unaware that this effect can be corrected for, or that this type of misleading claim is malpractice. But someone in the chain of publishing - the journal editors, the reviewers, the large author list, or Jakob Nielsen who is promoting this - should have caught this. It is their main result!<p>In the absence of legitimate statistics, the article&#x27;s circumstances point to a failure to detect measurable differences between fonts. There are two ways fonts may be better than each other:<p>1. across the population, so that one font is better for everyone<p>2. personalized, so that different fonts are better for different people<p>The first case should be easily detectable, and the second case should see some correlation between preference and speed because of a familiarity bias (<a href="https:&#x2F;&#x2F;dl.acm.org&#x2F;doi&#x2F;10.1145&#x2F;3502222#d1e7351" rel="nofollow">https:&#x2F;&#x2F;dl.acm.org&#x2F;doi&#x2F;10.1145&#x2F;3502222#d1e7351</a>). These are the Bayesian expectations I walked in with. Neither of these appear supported by the article, although I have only skimmed it.<p>To be clear, the experiment does not give evidence that fonts perform equally either. It looks more likely that the experiment design failed.