I’ve read the article you sent sort of carefully plus this one (https://www.doc.ic.ac.uk/~eedwards/compsys/float).
What I am trying to understand is when is adding two numbers result in errors. My understanding from the article is that 7.33 decimal places is what is “remembered” in the mantissa. But it is also tracking the floating point location with the exponent.
So my current understanding is that if both numbers are in the same exponent range then adding them is fine (as long as it’s not smaller than 7.3 decimals…and if has parts in that range then the computer wouldn’t even be able to store that part anyway).
So what I am struggling to really articulate is when we get into issues. If we have both number be (1.0 + 0.M)*2^-7 and (1.0+M)*2^4 range…would we get into problems?
I guess what I’m trying to understand is when underflows and overflows occur (and hoping that is the only type of error).
My loss is really small (from my synthetic data set), so it’s not a standard scenario.