The Fundamental Axiom of Floating Point Arithmetic
This “fundamental axiom” is not true:<p><i>”This means that for any two floating point numbers, say x and y, any operation involving them will give a floating point result which is within a factor of 1+ϵ of the true result.”</i><p>It fails for x-y when x and y are close. Subtraction is the easiest way to get big floating point errors. The article does mention this towards the end but you need to wade through a lot of equations before you get told this very important caveat.<p>The more tricky way to accumulate errors is to add lots of small numbers to a big number. Just summing a list of floats can be fraught with peril! Kahan’s summation algorithm is enlightening: <a href="https://en.wikipedia.org/wiki/Kahan_summation_algorithm" rel="nofollow">https://en.wikipedia.org/wiki/Kahan_summation_algorithm</a>