Demystifying Floating Point Precision
I&#x27;ve stated some notes on Floats before, and I think this blogpost is pretty good overall.<p>The one thing that trips up a lot of people (that wasn&#x27;t mentioned in this post) is that Floats are non-associative. Try the following in Python (or whatever language that uses Doubles):<p>&gt;&gt;&gt; 1 + 1 + (2. * * 53)<p>&gt; 9007199254740994.0<p>&gt;&gt;&gt; 1 + (1 + (2. * * 53))<p>&gt; 9007199254740992.0<p>claytonjy noted a point of confusion, the above statement has been edited for clarity.<p>The number 53 is chosen, because there&#x27;s 53 bits in the double mantissa. Which is when &quot;+1&quot; will become too small and drop-off of the Double.<p>In the first case, 1+1 becomes 2 (which is large enough to be added to 2. * * 53). So you end up with the &quot;correct&quot; answer. But (2. * * 53 + 1) rounds down (because the 1 drops off), and the additional +1 afterwards also drops off due to rounding error.<p>Therefore, Floating Point math is commutative but NOT Associative. And that&#x27;s really what makes things confusing for most people, in my experience.<p>x86 with the x87 coprocessor actually performs 80-bit floats to deal with this issue. So you can afford to lose a fair number of bits under older x86 platforms. But modern C++ &#x2F; C code typically compiles to the faster SSE2 instructions, which only keep 64-bits per operation. This only demonstrates that rounding behavior is not only architecture-specific, but also compiler-option specific !!