You Only Need to Test with 5 Users (2000)
This is an excellent point, and the much more fascinating corollaries:<p>- a product designer/manager of something with 1,000,000 users won’t learn more about usability than a product designer/manager of something with 15 users. All those measurements of flows and secret at scale analytics data is sort of worthless for the purposes of usability.<p>- people with 15 users worth of learning about usability instead of 0 users are way undervalued, while people with 1,000,000 are way overvalued<p>- “we don’t know until we test it,” the #1 refrain of big company design nowadays, is intellectually bankrupt for most free software, since if it looked bad for 15 users it’s probably going to still be bad for 1,000,000<p>- after you’ve shown something to 15 people and they don’t like it due to usability problems, you’re extremely unlikely to find 1,000,000 more who will like it.<p>This also appeals to my instinct that there is something learnable about design, that great design is achieved by people and not by massive datasets.