If 1+x is 1, how much is 1-x?
Nice post.<p>When people wonder why floating point calculations sometimes don't perfectly match the expectations of a "correct" answer[1], inevitably people will often respond with the famous paper, <i>"What Every Computer Scientist Should Know About Floating-Point Arithmetic"</i>[2].<p>Unfortunately, the people who suggest that paper don't realize that it's a very technical treatise and not an appropriate introduction for the types of programmers who ask the question.<p>I've always thought a better answer was to have the programmer "construct" a toy floating point format (e.g. 8 bits instead of 32-bit-or 64-bits).  The programmer would then notice that he has a finite representation of 256 possible "states" to represent -inf to 0 to +inf.  The programmer would have to <i>pick</i> a range and a precision to "squeeze" the Real Numbers into that representation of 8 bits.<p>Design choice 1:  I'll have my 256 possible states represent -1billion to +1billion.  Well, since you can't overlay 256 states/values across 2 billion <i>unique</i> values, you will have huge "gaps" between numbers.<p>Design choice 2:  I'll have my 256 possible states represent -10 to +10.  With a smaller range, you can now increase the precision and represent fractional numbers with digits after the decimal point.  But the range is very small.  Also, you still have "gaps" where you can't represent most[3] fractional numbers.<p>The programmer would quickly notice that he'd run into some contradictions.  No matter what he does, there will always be gaps.  The lightbulb would then go on and then he'd immediately scale up the limitations inherent in 8bit floating point all the way to 64bit floating point and know exactly why 0.3333 * 3.0 does not exactly equal 1.0.<p>It had been on my vague "todo" list to write such a tutorial but your blog post already gets that "construction of floating point" idea nicely.  The programmer can then explore more advanced topics of "normalization" and "bias" in designing a floating point format.<p>[1] <a href="http://stackoverflow.com/questions/588004/is-floating-point-math-broken" rel="nofollow">http://stackoverflow.com/questions/588004/is-floating-point-...</a><p>[2] <a href="http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html" rel="nofollow">http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.ht...</a><p>[3]<a href="http://math.stackexchange.com/questions/474415/intuitive-explanation-for-how-could-there-be-more-irrational-numbers-than-rati" rel="nofollow">http://math.stackexchange.com/questions/474415/intuitive-exp...</a>