Answers.Body: <p>Here are a few more ideas:</p>
  
  <p>Point out that that only $2^{32}$ values can be represented in 32 bits (or $2^{64}$ values in 64 bits), while there are an infinite number of real numbers. Therefore, not all real numbers can be represented exactly. For example, if you take the smallest positive number that can be represented exactly and divide it by 2, you can't represent it exactly (by definition).</p>
  
  <p>Explain that floating-point numbers are represented as the sum of powers of 2. Have them compute relatively easy values, such as $1\frac{3}{4} = 1 + \frac{1}{2} + \frac{1}{4}$. Ask them to represent $\frac{1}{3}$ as the sum of powers of 2. They will not be able to.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-26T00:28:29.307Z
Answers.LastActivityDate: 2017-06-26T14:05:08.933Z
Answers.LastEditDate: 2017-06-26T14:05:08.933Z
Answers.UserDisplayName: Ellen Spertus
Answers.Body: <p>As a student myself, I had a course about how processing and encoding works where we had a detailed part about operations in different bases and how <code>floats</code> and <code>doubles</code> differ from one another by the number of bytes allocated, and how the computer's different ways of encoding floating numbers can be done by hand to get a grasp of it, but this wasn't until my third year, after having a first introduction to operations in different bases during my second year.</p>
  
  <p>Only basic understanding of numbers in different bases and knowledge of the fact that <code>floats</code> are coded on a given number of bits is necessary to understand why it may be a problem, you could use the example of scientific calculations, where precision is important to make students get the stakes. But if you want students to really understand it fully, I think training operations in different bases and making them try by hand floating numbers coding and rounding may be necessary to get how frequent approximations are and how trivial calculations can give an answer different than what's expected, but more time consuming.</p>
  
  <p>Also, if you want a playful and yet well explained way to teach students how those imprecisions work and how it may affect programs, here is a 10 min long video about floating numbers imprecision in mario 64 that's really well explained : <a href="https://www.youtube.com/watch?v=9hdFG2GcNuA" rel="nofollow noreferrer">https://www.youtube.com/watch?v=9hdFG2GcNuA</a></p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-26T14:02:49.683Z
Answers.LastActivityDate: 2017-06-27T14:29:25.827Z
Answers.LastEditDate: 2017-06-27T14:29:25.827Z
Answers.UserDisplayName: Romain Bernard
Answers.Body: <p>There are plenty of good answers here, so I'll just add the extremely simple example I concocted to show why addition is not associative.</p>
  <p>Suppose your have a decimal floating point system with one significant digit. Now what is <code>4+6+7</code>? If you evaluate left to right it's <code>2 * 10^1</code>, if you evaluate right to left it's <code>1 * 10^1</code>. (This is with rounding. I give my students a bonus exercise of coming up with an example that uses truncation.)</p>
  <p>Worked out: <a href="https://pages.tacc.utexas.edu/%7Eeijkhout/istc/html/arithmetic.html#Associativity" rel="nofollow noreferrer">https://pages.tacc.utexas.edu/~eijkhout/istc/html/arithmetic.html#Associativity</a>  and read the whole page for anything else you want to know about floating point math.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-12-27T17:48:43.140Z
Answers.LastActivityDate: 2021-12-27T19:45:05.540Z
Answers.LastEditDate: 2021-12-27T19:45:05.540Z
Answers.UserDisplayName: Victor Eijkhout
Answers.Body: <p>If your students can understand why:</p>
  
  <p>$$\frac{1}{3} = 0.\dot{3}$$</p>
  
  <p>then you can explain simply why floating point calculations are "broken".</p>
  
  <p>When you divide one by three on a simple calculator, you can observe that it usually shows 0.333333333... until it runs out of digits (unless you have a more advanced calculator that can show recurring digits like <a href="https://i.ytimg.com/vi/BF74WoLp6NE/hqdefault.jpg" rel="noreferrer">this</a>).</p>
  
  <p>You can explain that the phenomenon observed here is because of the same thing: one third in floating point is an <strong>infinitely repeating</strong> sequence:</p>
  
  <p>$$ \frac{1}{3} = 0.0\overline{10}_2$$</p>
  
  <p>If we could store that fully, then multiplying it by three would indeed give us one, as expected. But, like our naive calculator, we don't store all the bits in the sequence, so you end up with a slight inaccuracy. Pointing out that
  $ 0.333333333 \times 3 = 0.999999999$ should be enough to illustrate the point that with an infinitely repeating decimal, you will not get the correct answer unless all the digits are taken account of (and you'll always be ever so slightly off the correct value, no matter which approximation you use).</p>
  
  <p>It should then be clear to them that unless you use the exact value of $\frac{1}{3}$ in the binary representation, you will also get inaccuracies.</p>
  
  <p>Something to note is that <strong>a finite decimal may be a repeating binary fraction</strong> (a significant example is $\frac{1}{10}$, which is $0.1_{10}$, but $0.0\overline{0011}_2$), but <a href="https://math.stackexchange.com/a/261989/378985">not vice versa</a>. Clearly, because of this, seemingly finite decimal numbers added together can lead to an infinite binary fraction, and hence an imprecise answer.</p>
  
  <p>If your students are also familiar with <strong>scientific notation</strong>, then explaining the floating point system of a significant and mantissa should also be relatively painless. Since floating point can represent both <strong>very small</strong> and <strong>very large</strong> numbers, representing numbers as $\text{significand} \times 2^{\text{exponent}}$ is the most flexible option. You need not elaborate much further than that, though, I don't think. As <a href="https://cseducators.stackexchange.com/a/1100/2">Ben I.</a> mentions, a functional knowledge of floating point is needed, but the internals are probably not needed at this stage.</p>
  
  <p>I don't see any reason why you <em>shouldn't</em> introduce the reason why floating point numbers are inaccurate to them quite early on — it's probably worse <em>not</em> to tell them, and leave them to be <strong>very</strong> confused when their program doesn't work because of the tiny inaccuracies ruining comparisons.</p>
  
  <p>I think <a href="http://floating-point-gui.de/basic/" rel="noreferrer">floating-point-gui.de</a> does a great job at explaining, very concisely, without any implementation details (and you can read on about the details <strong>if you want</strong>). Using that as a resource may be helpful to find the right level to teach.</p>
  
  <p>If you want to give your students something to play around with and get an understanding of the format of IEEE 754 floating point, you could try <a href="https://www.h-schmidt.net/FloatConverter/IEEE754.html" rel="noreferrer">this floating point number converter</a> which will show the representation of any number you enter.</p>
  
  <p>It might also be wise to introduce a strategy to solve this problem—<a href="http://floating-point-gui.de/errors/comparison/" rel="noreferrer">epsilon values</a>. It should be relatively straightforward to explain that you just compare if a number is 'close' to what you expected with a floating point operation, rather than exactly equal; there are plenty of resources available that explain epsilon values further if necessary.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-25T20:34:35.650Z
Answers.Text: “If your students can understand why $1/3 = 0.\dot 3$” — that's actually a difficult one. I expect that most students don't really understand it.
Answers.UserDisplayName: Gilles 'SO- stop being evil'
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-26T08:15:17.260Z
Answers.Text: @Giles It's a very long time since I first met recurring decimals, but I don't recall too many fellow students not getting it.
Answers.UserDisplayName: TripeHound
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-26T08:19:19.420Z
Answers.Text: Once you've shown the problem with 1/3, it might be better to show 1/10 in binary. Although, as noted above, I could "get" recurring decimals, there's a danger of people thinking it's something peculiar about _one third_. Seeing the same problem happening with a "safe" number like _one tenth_ drove the point home for binary representations.
Answers.UserDisplayName: TripeHound
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-26T12:52:53.247Z
Answers.Text: +1, comment by @TripeHound about the dangers of "safe"/not-periodic numbers like 1/10 should be added to the answer. Also, [the problem with subtraction](https://stackoverflow.com/a/27539234/4209853)
Answers.UserDisplayName: xDaizu
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-26T14:26:34.007Z
Answers.Text: I would respectfully suggest that **this is a very misleading** example to illustrate the **computer science problem** of floating point precision. By using a repeating decimal as your example, you are (incorrectly) turning this into the math problem of representing fractions as decimals rather than a computer science problem that 1.0f + 1.0f =! 2.0f. This will leave students with a lot of bugs and head scratching trying to figure out why this doesn't work (i.e. not understanding how floating point numbers are represented).
Answers.UserDisplayName: Robert Cartaino
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-26T15:53:05.093Z
Answers.Text: @RobertCartaino You have an excellent point that using a number that is not finite in base-10 could lead to some false assumptions. I used the example from the question ($\frac{1}{3}$) when writing this, but I can see how it might be confusing. I think you cannot escape the fact that some numbers cannot be represented in binary, leading to these errors, which is **inherently a mathematical issue**, but I've added some more explicit information on how to solve this problem. Is there anything specifically that you feel ought to be mentioned here that I missed?
Answers.UserDisplayName: Aurora0001
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-26T19:11:42.070Z
Answers.Text: "a finite decimal may be a repeating binary fraction **and vice versa**" – Actually, a finite binary fraction will always convert to a finite decimal fraction (with at most the same number of digits).
Answers.UserDisplayName: Paŭlo Ebermann
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-26T19:15:29.440Z
Answers.Text: @PaŭloEbermann You're right. Thanks for pointing that out. I've amended that (and found a proof on Math.SE for anyone interested).
Answers.UserDisplayName: Aurora0001
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-26T19:39:40.630Z
Answers.Text: Something I'd suggest adding as an exercise with a regular calculator is that you can see the extra digits it hides and then uses in rounding to make everything seem to work out as expected.  Ex if after entering `1/7` your calculator displays `0.14285714", you can hit subtract and type in `0.142857141` hit enter and it'll show something like `2.8571E-9`.  With the exact amount depending on the calculators internal representation.
Answers.UserDisplayName: Dan Is Fiddling By Firelight
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-26T19:41:08.450Z
Answers.Text: @RobertCartaino I do not understand your objection here. 1.0 + 1.0 _does_ equal 2.0 in common floating-point systems. The problem is instead with 1.0 / 10.0 * 10.0, which rounds well in decimal but not in binary, and this answer explains why. Were you commenting on an earlier edit?
Answers.UserDisplayName: B. Szonye
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-27T11:09:06.217Z
Answers.Text: @Gilles - where in the world is this true? I'm in europe and that is literally a grade school material. The only way someone might not understand that is if they have a mental handicap.
Answers.UserDisplayName: Davor
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-27T11:17:17.113Z
Answers.Text: @Davor There's a huge gap between vaguely understanding that $1/3$ is not equal to $0.33$ and understanding infinitary decimal representation. Understanding that $0.\bar 9 = 1$ (as opposed to $0.\bar 9 \approx 1$) is a very difficult milestone. Having heard it in school is not the same thing as understanding it.
Answers.UserDisplayName: Gilles 'SO- stop being evil'
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-12-27T17:42:55.000Z
Answers.Text: Have you tried lately whether `3 * (1/3) -1` was zero? Current processors all have FMA units, which make this expression come out to zero! You need to do `3*3*(1/3)*(1/3)-1` to get a nonzero result.
Answers.UserDisplayName: Victor Eijkhout
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-25T20:11:04.230Z
Answers.LastActivityDate: 2017-06-26T19:14:21.367Z
Answers.LastEditDate: 2017-06-26T19:14:21.367Z
Answers.UserDisplayName: Aurora0001
Answers.Body: <p>Even fairly early coders need to understand some limited form of why </p>
  
  <pre><code>    double a = 1.000001;
      double b = 0.000001;
      System.out.println((a-b)==1.0);
  </code></pre>
  
  <p>ultimately prints <code>false</code>, and how to avoid this problem.</p>
  
  <p>However, a more serious understanding of <strong>IEEE 754</strong> can wait.  I have my kids implement this standard (just the number representation, no arithmetic) as an elective after they have already learned about two's complement, big and little endian, and how a logical-gate 8-bit adder works.  Before they've had some time to live with and understand how binary numbers work, they won't have much of a chance of appreciating how floating point is dealt with on any serious level.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-25T19:27:50.090Z
Answers.LastActivityDate: 2017-06-25T19:27:50.090Z
Answers.UserDisplayName: Ben I.
Answers.Body: <p>A few points:</p>
  
  <ul>
  <li>Do not call the discrepancies rounding errors</li>
  <li>Do not confuse students with epsilon math</li>
  <li>Do not obsess on teaching them exactly how IEEE floats are stored</li>
  <li>Do not show them examples of the problem without explaining why things happened the way they did</li>
  </ul>
  
  <p>Teach them this:</p>
  
  <p>Floats are confusing precisely because the way they are stored and the way they are represented is incompatible.  The values don't always translate 1 to 1. Some values that are easy to represent can't be stored faithfully with finite memory. So they are converted (not rounded) to the closest storable value available. Sometimes this goes completely unnoticed because when that stored value is converted (not rounded) back to a representation it ends up being the same representation it had before it went in. However, if math is done on this value then it's done on the stored value. The new stored value might get lucky and convert back to the correct representation. It might not. It's this "might" thing that makes this so hard. I can run tests on numbers thinking I got back what I put in so it must store exactly only to have my heart broken when I do math with it.</p>
  
  <p>For example:</p>
  
  <pre><code>1.03 - .42
  </code></pre>
  
  <p>gives</p>
  
  <pre><code>0.6100000000000001
  </code></pre>
  
  <p>On their own 1.03 and .42 convert in and out of memory fine despite not being perfectly stored. Their difference is ALMOST right. It just doesn't happen to convert the way we hoped. Other math can put us at this same stored value that expects exactly this representation. There's no perfect way to round it. We simply don't know any more at this point than what was stored. </p>
  
  <p>Yet:</p>
  
  <pre><code>.01 - .02
  </code></pre>
  
  <p>gives a perfect</p>
  
  <pre><code>-0.01
  </code></pre>
  
  <p>Why? Because we got lucky. The information that was lost in conversion just happened to not be enough to matter once it was converted back.</p>
  
  <p>I hate problems that are hard to diagnose because I keep getting "lucky".</p>
  
  <p>Now you can throw a lot of math and diagrams at this problem trying to explain how floats are stored but the thing to teach is how to know which numbers can't be stored perfectly.  Because the computer isn't going to tell you.</p>
  
  <p>It's the students expectation of how floats will behave that you want to cultivate. They shouldn't expect them to behave like calculators. They shouldn't expect them to be useless. They should expect if you can't get to a number by addition and dividing by 2 that the numbers are going to be fuzzy. That's fine for scientific measurements that just want to be close. Not good for accounting that cares as much about pennies as dollars. A base 10 currency won't always store precisely. Though if you can find a currency that is always divided by 2 then floats will keep your accountant happy.</p>
  
  <p><a href="https://i.sstatic.net/2j72n.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/2j72n.png" alt="ruler with inches"></a></p>
  
  <p>Show them a ruler with inches on it. Floats can store inch fractions  perfectly. This should debunk the myth that floats have rounding errors. Entering these into a computer and doing math on them is a good exercise because it forces the student to do the conversion by hand. These conversions will all happen without discrepancies. You can even challenge the students to try to catch the computer doing inch fraction math wrong.</p>
  
  <p>I used to teach this with drafting rulers. You can get one that has decimal inches. Put the decimal inch ruler next to the fraction inch ruler. Show them how some numbers match up but others don't.  The decimal inch ruler represents what you see and what you type. The fraction inch ruler represents what is stored.</p>
  
  <p><a href="https://i.sstatic.net/R146f.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/R146f.png" alt="enter image description here"></a></p>
  
  <p>You can even get students to reproduce the same kinds of errors by having them take measurements of values from the decimal ruler on the fraction ruler and trying to convert back. They'll quickly notice that some convert back easily while others don't. Well, as long as you don't let them simply remember how the story started.</p>
  
  <p>You can find links to other resources in my earlier posts <a href="https://softwareengineering.stackexchange.com/a/344327/131624">here</a>. </p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-27T07:24:30.050Z
Answers.Text: I like how you stress the difference between rounding and conversion. However, your use of terminology is slightly incorrect compared to IEEE754. Conversion is the process of switching between decimal and float. In some cases, switching back and forth will not lose information. However, there is usually still rounding taking place in those conversions, possibly even if after converting back you end up with the same result (the two roundings simply cancelled out in that case). I would be hesitant to oversimplify things here, as that might do more harm than good in the long run.
Answers.UserDisplayName: ComicSansMS
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-27T08:24:59.203Z
Answers.Text: @ComicSansMS I'm trying to get away from the myth that the discrepancies in floats are just rounding errors.  I've seen to many students turn in code that attempts to correct the problem with their own rounding code. That's the oversimplification to avoid.
Answers.UserDisplayName: candied_orange
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-27T13:45:11.420Z
Answers.Text: The decimal inch ruler is perfect here.
Answers.UserDisplayName: Sean Houlihane
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-26T18:57:39.607Z
Answers.LastActivityDate: 2017-06-27T16:11:50.217Z
Answers.LastEditDate: 2017-06-27T16:11:50.217Z
Answers.UserDisplayName: candied_orange
Answers.Body: <p>In my experience, the most confusing aspect here is not that there are rounding errors. Students should already have an understanding that there are physical limits that keep us from calculating with infinite precision all the time. What is confusing though is that these rounding errors occur in cases that might seem trivial to students.</p>
  
  <p>What is important to understand here is that there are real numbers which do have a finite representation in decimal, but lead to an infinite representation when being converted to binary. <code>0.1</code> is probably the canonical example here. 
  Students are used to thinking in base 10, so the most surprising rounding errors are those, where the operation still stays within the number of a couple of significant decimal digits, and therefore looks completely harmless when writing it down on paper, but still blows up because the numbers cannot be represented exactly in binary. Think: $0.1 + 0.2 - 0.3 \neq 0$.</p>
  
  <p>Hammering this single point home should be enough to convince students that floats can lead to surprising rounding errors. The more challenging task is then to restore their faith in floats by explaining that the behavior of floats is still completely deterministic and one can actually give exact bounds for the expected rounding errors produced by a specific calculation.</p>
  
  <p>For instance, many programming languages these days have support for <a href="https://stackoverflow.com/questions/4825824/hexadecimal-floating-constant-in-c">hex float literals</a>, which allow specifying float literals in base 16, eliminating potential rounding errors when converting the base 10 numbers from the source code to the base 2 numbers used by the machine. You still can get rounding errors on each calculation, but knowing what the numbers look like to the machine makes it much more predictable when to expect such rounding errors. Also, most of these calculation rounding errors are actually not due to the machine being unable to calculate exactly, but rather the fact that the result of the calculation can not be represented exactly in float. So the result is often really the closest to the correct result that you can possibly achieve with the chosen representation.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-26T09:31:48.580Z
Answers.Text: Firstly, I'd like to welcome you to Computer Science Educators! Secondly, this answer is superb, I do hope we will be hearing more from you.
Answers.UserDisplayName: ItamarG3
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-26T13:49:09.177Z
Answers.Text: I joined CSE just to up-vote this...
Answers.UserDisplayName: WLGfx
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-27T09:47:04.117Z
Answers.Text: If you need accuracy beyond the float standard, use integer math or a data type like [Decimal](https://stackoverflow.com/questions/1056323/difference-between-numeric-float-and-decimal-in-sql-server).
Answers.UserDisplayName: Cees Timmerman
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-12-28T15:57:05.940Z
Answers.Text: Nice answer, but have you actually tried whether `.1+.2 ?=? .3`? With clang11, putting the expression literally in a program the compiler probably optimizes it away, and yes, it is equal. I had to write `pointone + .2f -.3f` where `pointone` is read from `cin` to get a nonzero result. And even then, you only get nonzero with 2 out of the 4 rounding modes. And not the default one. So: Test Every Assertion You Make. This Crap Is Subtle! (My latest learning experience: `a * 1/a - 1` is zero on modern processors because this is an FMA instruction, which has single rounding.)
Answers.UserDisplayName: Victor Eijkhout
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-26T08:09:46.570Z
Answers.LastActivityDate: 2017-06-26T08:34:05.167Z
Answers.LastEditDate: 2017-06-26T08:34:05.167Z
Answers.UserDisplayName: ComicSansMS
Body: <p>When I first started working with floating point arithmetic, issues like the one illustrated in the sample below both frustrated and confused me:</p>
  
  <pre><code>scala&gt; (1f/3)*3d // admittedly a bit contrived using 1f
  res0: Double = 1.0000000298023224
  scala&gt; res0 == 1.0
  res1: Boolean = false
  </code></pre>
  
  <p>I was used to mathematics, and a calculator which used fixed point arithmetic - an environment in which the answers were always "right" (to a given number of decimal places)! The way most programming languages handled decimal numbers felt <em>broken</em> and almost certainly useless for storing monetary values or for use in scientific code. I didn't appreciate the trade-off being made in terms of speed and space by taking the approach IEEE 754 did until much later on when I looked at the specification in detail and the analogy with standard scientific form was brought up.</p>
  
  <p>At what stage should students be introduced to how floating point numbers are stored, and (more importantly) how can the motivations for this approach be properly conveyed?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-04-18T11:06:15.347Z
Comments.Text: What "inaccuracies?" You mean, results from erroneously deisgned chips? No? There are no "inaccuracies". So lesson number one is to not rely on thoughtless presumptions but read the specification for what you're using & lesson number 2 is there are two different abstractions involved in programming with floating point.
Comments.UserDisplayName: philipxy
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-04-18T12:22:09.767Z
Comments.Text: PS Show (better, have them graph/plot) an actual (normalized) float number line with its idiosyncratic spacing: `..._|______________|______________|______|_______|________|_______|_______|_______|_______|___|___|___|___|___|___|___|_|_|_|_|_|_|_||||||||________0________|||||||||_|_|_|_|_|_|_|___|___|___|___|___|___|___|______|_______|________|_______|_______|_______|_______|______________|______________|_...` [(1st figure in section 3_2 here)](http://www.toves.org/books/float/) What more need one say?
Comments.UserDisplayName: philipxy
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-06-26T00:17:47.157Z
Comments.Text: Here are some fun SMBC cartoons on the topic. http://www.smbc-comics.com/comic/2008-03-16 http://www.smbc-comics.com/?id=2999
Comments.UserDisplayName: Ellen Spertus
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-06-26T15:39:17.217Z
Comments.Text: Here is a YouTube video that gives an incredible explanation of floats: https://www.youtube.com/watch?v=9hdFG2GcNuA It's a speedrunner playing Super Mario 64 in an emulator, going into detail about how floating point numbers give rise to some unusual glitches in the game. The video is both educational and genuinely fascinating.
Comments.UserDisplayName: Kevin
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-06-26T17:56:07.450Z
Comments.Text: I realize that this doesn't change the fact that you need to explain float behavior to students but you should also consider the way BigDecimal works--it shouldn't have the same type of binary rounding problems because it bypasses the whole thing by using Integers for all decimal calculations.
Comments.UserDisplayName: Bill K
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-06-26T19:09:06.237Z
Comments.Text: @BillK the example in the question won't be better with BigDecimal. (Depending on how the division is done, you might get an exception there. But 1/3 is not representable exactly as a decimal number either.)
Comments.UserDisplayName: Paŭlo Ebermann
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-06-26T19:36:19.480Z
Comments.Text: How do you teach them the analogous **integer** problem (here, in C): `int i = 1/3; i = 3 * i; printf("%d\n", i);` -- surprise, the result is 0.
Comments.UserDisplayName: Pete Becker
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-06-26T19:43:05.487Z
Comments.Text: Actually I believe it will be better since it would be very obvious and predictable if the code fiddled with the precision (currently it casts it to float and double).  BigDecimal can't represent 1/3 exactly, but if you set it to, say 10 decimal places then every version of 1/3-1/3 should equal 0.  If you set one to 3 decimal places and the other to 4, you should always get a very predictable and sensible .0003.  The best part is you won't ever get rounding problems on numbers that shouldn't have rounding problems like .1
Comments.UserDisplayName: Bill K
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-06-27T08:39:14.297Z
Comments.Text: A useful model that keeps most of the floating point problems is a computer that works with two decimal places. So, you can exactly represent 75 and 0.12, but 114 is rounded to 110. Then you can go through practical examples, like finding the middle point between 92 and 94, that in the naive implementation is 95.
Comments.UserDisplayName: Davidmh
Tags: <number-system>
Title: How should floating point inaccuracies be explained and justified?
ThreadText: # How should floating point inaccuracies be explained and justified?
  
  (Asked by: username_0 on 2017-06-25)
  
  When I first started working with floating point arithmetic, issues like the one illustrated in the sample below both frustrated and confused me:
  
  ```
  scala> (1f/3)*3d // admittedly a bit contrived using 1f
  res0: Double = 1.0000000298023224
  scala> res0 == 1.0
  res1: Boolean = false
  
  ```
  
  I was used to mathematics, and a calculator which used fixed point arithmetic - an environment in which the answers were always "right" (to a given number of decimal places)! The way most programming languages handled decimal numbers felt *broken* and almost certainly useless for storing monetary values or for use in scientific code. I didn't appreciate the trade-off being made in terms of speed and space by taking the approach IEEE 754 did until much later on when I looked at the specification in detail and the analogy with standard scientific form was brought up.
  
  At what stage should students be introduced to how floating point numbers are stored, and (more importantly) how can the motivations for this approach be properly conveyed?
  
  **Tags:** number-system
  
  > **Comments:**
  > **username_2**: Here are some fun SMBC cartoons on the topic. http://www.smbc-comics.com/comic/2008-03-16 http://www.smbc-comics.com/?id=2999  
  > **username_3**: Here is a YouTube video that gives an incredible explanation of floats: https://www.youtube.com/watch?v=9hdFG2GcNuA It's a speedrunner playing Super Mario 64 in an emulator, going into detail about how floating point numbers give rise to some unusual glitches in the game. The video is both educational and genuinely fascinating.  
  > **username_4**: I realize that this doesn't change the fact that you need to explain float behavior to students but you should also consider the way BigDecimal works--it shouldn't have the same type of binary rounding problems because it bypasses the whole thing by using Integers for all decimal calculations.  
  > **username_5**: @username_4 the example in the question won't be better with BigDecimal. (Depending on how the division is done, you might get an exception there. But 1/3 is not representable exactly as a decimal number either.)  
  > **username_6**: How do you teach them the analogous **integer** problem (here, in C): `int i = 1/3; i = 3 * i; printf("%d\n", i);` -- surprise, the result is 0.  
  > **username_4**: Actually I believe it will be better since it would be very obvious and predictable if the code fiddled with the precision (currently it casts it to float and double).  BigDecimal can't represent 1/3 exactly, but if you set it to, say 10 decimal places then every version of 1/3-1/3 should equal 0.  If you set one to 3 decimal places and the other to 4, you should always get a very predictable and sensible .0003.  The best part is you won't ever get rounding problems on numbers that shouldn't have rounding problems like .1  
  > **username_7**: A useful model that keeps most of the floating point problems is a computer that works with two decimal places. So, you can exactly represent 75 and 0.12, but 114 is rounded to 110. Then you can go through practical examples, like finding the middle point between 92 and 94, that in the naive implementation is 95.  
  > **username_1**: What "inaccuracies?" You mean, results from erroneously deisgned chips? No? There are no "inaccuracies". So lesson number one is to not rely on thoughtless presumptions but read the specification for what you're using & lesson number 2 is there are two different abstractions involved in programming with floating point.  
  > **username_1**: PS Show (better, have them graph/plot) an actual (normalized) float number line with its idiosyncratic spacing: `..._|______________|______________|______|_______|________|_______|_______|_______|_______|___|___|___|___|___|___|___|_|_|_|_|_|_|_||||||||________0________|||||||||_|_|_|_|_|_|_|___|___|___|___|___|___|___|______|_______|________|_______|_______|_______|_______|______________|______________|_...` [(1st figure in section 3_2 here)](http://www.toves.org/books/float/) What more need one say?  
  
  ---
  
  ## 7 answers
  
  ### Answer by username_10 on 2017-06-25. Score: 25 | Accepted answer
  
  If your students can understand why:
  
  $$\frac{1}{3} = 0.\dot{3}$$
  
  then you can explain simply why floating point calculations are "broken".
  
  When you divide one by three on a simple calculator, you can observe that it usually shows 0.333333333... until it runs out of digits (unless you have a more advanced calculator that can show recurring digits like [this](https://i.ytimg.com/vi/BF74WoLp6NE/hqdefault.jpg)).
  
  You can explain that the phenomenon observed here is because of the same thing: one third in floating point is an **infinitely repeating** sequence:
  
  $$ \frac{1}{3} = 0.0\overline{10}\_2$$
  
  If we could store that fully, then multiplying it by three would indeed give us one, as expected. But, like our naive calculator, we don't store all the bits in the sequence, so you end up with a slight inaccuracy. Pointing out that
  $ 0.333333333 \times 3 = 0.999999999$ should be enough to illustrate the point that with an infinitely repeating decimal, you will not get the correct answer unless all the digits are taken account of (and you'll always be ever so slightly off the correct value, no matter which approximation you use).
  
  It should then be clear to them that unless you use the exact value of $\frac{1}{3}$ in the binary representation, you will also get inaccuracies.
  
  Something to note is that **a finite decimal may be a repeating binary fraction** (a significant example is $\frac{1}{10}$, which is $0.1\_{10}$, but $0.0\overline{0011}\_2$), but [not vice versa](https://math.stackexchange.com/a/261989/378985). Clearly, because of this, seemingly finite decimal numbers added together can lead to an infinite binary fraction, and hence an imprecise answer.
  
  If your students are also familiar with **scientific notation**, then explaining the floating point system of a significant and mantissa should also be relatively painless. Since floating point can represent both **very small** and **very large** numbers, representing numbers as $\text{significand} \times 2^{\text{exponent}}$ is the most flexible option. You need not elaborate much further than that, though, I don't think. As [username_18.](https://cseducators.stackexchange.com/a/1100/2) mentions, a functional knowledge of floating point is needed, but the internals are probably not needed at this stage.
  
  I don't see any reason why you *shouldn't* introduce the reason why floating point numbers are inaccurate to them quite early on — it's probably worse *not* to tell them, and leave them to be **very** confused when their program doesn't work because of the tiny inaccuracies ruining comparisons.
  
  I think [floating-point-gui.de](http://floating-point-gui.de/basic/) does a great job at explaining, very concisely, without any implementation details (and you can read on about the details **if you want**). Using that as a resource may be helpful to find the right level to teach.
  
  If you want to give your students something to play around with and get an understanding of the format of IEEE 754 floating point, you could try [this floating point number converter](https://www.h-schmidt.net/FloatConverter/IEEE754.html) which will show the representation of any number you enter.
  
  It might also be wise to introduce a strategy to solve this problem—[epsilon values](http://floating-point-gui.de/errors/comparison/). It should be relatively straightforward to explain that you just compare if a number is 'close' to what you expected with a floating point operation, rather than exactly equal; there are plenty of resources available that explain epsilon values further if necessary.
  
  > **Comments:**
  > **username_11**: “If your students can understand why $1/3 = 0.\dot 3$” — that's actually a difficult one. I expect that most students don't really understand it.  
  > **username_12**: @Giles It's a very long time since I first met recurring decimals, but I don't recall too many fellow students not getting it.  
  > **username_12**: Once you've shown the problem with 1/3, it might be better to show 1/10 in binary. Although, as noted above, I could "get" recurring decimals, there's a danger of people thinking it's something peculiar about _one third_. Seeing the same problem happening with a "safe" number like _one tenth_ drove the point home for binary representations.  
  > **username_13**: +1, comment by @username_12 about the dangers of "safe"/not-periodic numbers like 1/10 should be added to the answer. Also, [the problem with subtraction](https://stackoverflow.com/a/27539234/4209853)  
  > **username_14**: I would respectfully suggest that **this is a very misleading** example to illustrate the **computer science problem** of floating point precision. By using a repeating decimal as your example, you are (incorrectly) turning this into the math problem of representing fractions as decimals rather than a computer science problem that 1.0f + 1.0f =! 2.0f. This will leave students with a lot of bugs and head scratching trying to figure out why this doesn't work (i.e. not understanding how floating point numbers are represented).  
  > **username_10**: @username_14 You have an excellent point that using a number that is not finite in base-10 could lead to some false assumptions. I used the example from the question ($\frac{1}{3}$) when writing this, but I can see how it might be confusing. I think you cannot escape the fact that some numbers cannot be represented in binary, leading to these errors, which is **inherently a mathematical issue**, but I've added some more explicit information on how to solve this problem. Is there anything specifically that you feel ought to be mentioned here that I missed?  
  > **username_5**: "a finite decimal may be a repeating binary fraction **and vice versa**" – Actually, a finite binary fraction will always convert to a finite decimal fraction (with at most the same number of digits).  
  > **username_10**: @username_5 You're right. Thanks for pointing that out. I've amended that (and found a proof on Math.SE for anyone interested).  
  > **username_15**: Something I'd suggest adding as an exercise with a regular calculator is that you can see the extra digits it hides and then uses in rounding to make everything seem to work out as expected.  Ex if after entering `1/7` your calculator displays `0.14285714", you can hit subtract and type in `0.142857141` hit enter and it'll show something like `2.8571E-9`.  With the exact amount depending on the calculators internal representation.  
  > **username_16**: @username_14 I do not understand your objection here. 1.0 + 1.0 _does_ equal 2.0 in common floating-point systems. The problem is instead with 1.0 / 10.0 * 10.0, which rounds well in decimal but not in binary, and this answer explains why. Were you commenting on an earlier edit?  
  > **username_17**: @username_11 - where in the world is this true? I'm in europe and that is literally a grade school material. The only way someone might not understand that is if they have a mental handicap.  
  > **username_11**: @username_17 There's a huge gap between vaguely understanding that $1/3$ is not equal to $0.33$ and understanding infinitary decimal representation. Understanding that $0.\bar 9 = 1$ (as opposed to $0.\bar 9 \approx 1$) is a very difficult milestone. Having heard it in school is not the same thing as understanding it.  
  > **username_9**: Have you tried lately whether `3 * (1/3) -1` was zero? Current processors all have FMA units, which make this expression come out to zero! You need to do `3*3*(1/3)*(1/3)-1` to get a nonzero result.  
  
  ---
  
  ### Answer by username_20 on 2017-06-26. Score: 23
  
  In my experience, the most confusing aspect here is not that there are rounding errors. Students should already have an understanding that there are physical limits that keep us from calculating with infinite precision all the time. What is confusing though is that these rounding errors occur in cases that might seem trivial to students.
  
  What is important to understand here is that there are real numbers which do have a finite representation in decimal, but lead to an infinite representation when being converted to binary. `0.1` is probably the canonical example here.
  Students are used to thinking in base 10, so the most surprising rounding errors are those, where the operation still stays within the number of a couple of significant decimal digits, and therefore looks completely harmless when writing it down on paper, but still blows up because the numbers cannot be represented exactly in binary. Think: $0.1 + 0.2 - 0.3 \neq 0$.
  
  Hammering this single point home should be enough to convince students that floats can lead to surprising rounding errors. The more challenging task is then to restore their faith in floats by explaining that the behavior of floats is still completely deterministic and one can actually give exact bounds for the expected rounding errors produced by a specific calculation.
  
  For instance, many programming languages these days have support for [hex float literals](https://stackoverflow.com/questions/4825824/hexadecimal-floating-constant-in-c), which allow specifying float literals in base 16, eliminating potential rounding errors when converting the base 10 numbers from the source code to the base 2 numbers used by the machine. You still can get rounding errors on each calculation, but knowing what the numbers look like to the machine makes it much more predictable when to expect such rounding errors. Also, most of these calculation rounding errors are actually not due to the machine being unable to calculate exactly, but rather the fact that the result of the calculation can not be represented exactly in float. So the result is often really the closest to the correct result that you can possibly achieve with the chosen representation.
  
  > **Comments:**
  > **username_22**: Firstly, I'd like to welcome you to Computer Science Educators! Secondly, this answer is superb, I do hope we will be hearing more from you.  
  > **username_23**: I joined CSE just to up-vote this...  
  > **username_24**: If you need accuracy beyond the float standard, use integer math or a data type like [Decimal](https://stackoverflow.com/questions/1056323/difference-between-numeric-float-and-decimal-in-sql-server).  
  > **username_9**: Nice answer, but have you actually tried whether `.1+.2 ?=? .3`? With clang11, putting the expression literally in a program the compiler probably optimizes it away, and yes, it is equal. I had to write `pointone + .2f -.3f` where `pointone` is read from `cin` to get a nonzero result. And even then, you only get nonzero with 2 out of the 4 rounding modes. And not the default one. So: Test Every Assertion You Make. This Crap Is Subtle! (My latest learning experience: `a * 1/a - 1` is zero on modern processors because this is an FMA instruction, which has single rounding.)  
  
  ---
  
  ### Answer by username_18 on 2017-06-25. Score: 9
  
  Even fairly early coders need to understand some limited form of why
  
  ```
      double a = 1.000001;
      double b = 0.000001;
      System.out.println((a-b)==1.0);
  
  ```
  
  ultimately prints `false`, and how to avoid this problem.
  
  However, a more serious understanding of **IEEE 754** can wait. I have my kids implement this standard (just the number representation, no arithmetic) as an elective after they have already learned about two's complement, big and little endian, and how a logical-gate 8-bit adder works. Before they've had some time to live with and understand how binary numbers work, they won't have much of a chance of appreciating how floating point is dealt with on any serious level.
  
  
  ---
  
  ### Answer by username_19 on 2017-06-26. Score: 9
  
  A few points:
  
  * Do not call the discrepancies rounding errors
  * Do not confuse students with epsilon math
  * Do not obsess on teaching them exactly how IEEE floats are stored
  * Do not show them examples of the problem without explaining why things happened the way they did
  
  Teach them this:
  
  Floats are confusing precisely because the way they are stored and the way they are represented is incompatible. The values don't always translate 1 to 1. Some values that are easy to represent can't be stored faithfully with finite memory. So they are converted (not rounded) to the closest storable value available. Sometimes this goes completely unnoticed because when that stored value is converted (not rounded) back to a representation it ends up being the same representation it had before it went in. However, if math is done on this value then it's done on the stored value. The new stored value might get lucky and convert back to the correct representation. It might not. It's this "might" thing that makes this so hard. I can run tests on numbers thinking I got back what I put in so it must store exactly only to have my heart broken when I do math with it.
  
  For example:
  
  ```
  1.03 - .42
  
  ```
  
  gives
  
  ```
  0.6100000000000001
  
  ```
  
  On their own 1.03 and .42 convert in and out of memory fine despite not being perfectly stored. Their difference is ALMOST right. It just doesn't happen to convert the way we hoped. Other math can put us at this same stored value that expects exactly this representation. There's no perfect way to round it. We simply don't know any more at this point than what was stored.
  
  Yet:
  
  ```
  .01 - .02
  
  ```
  
  gives a perfect
  
  ```
  -0.01
  
  ```
  
  Why? Because we got lucky. The information that was lost in conversion just happened to not be enough to matter once it was converted back.
  
  I hate problems that are hard to diagnose because I keep getting "lucky".
  
  Now you can throw a lot of math and diagrams at this problem trying to explain how floats are stored but the thing to teach is how to know which numbers can't be stored perfectly. Because the computer isn't going to tell you.
  
  It's the students expectation of how floats will behave that you want to cultivate. They shouldn't expect them to behave like calculators. They shouldn't expect them to be useless. They should expect if you can't get to a number by addition and dividing by 2 that the numbers are going to be fuzzy. That's fine for scientific measurements that just want to be close. Not good for accounting that cares as much about pennies as dollars. A base 10 currency won't always store precisely. Though if you can find a currency that is always divided by 2 then floats will keep your accountant happy.
  
  [![ruler with inches](https://i.sstatic.net/2j72n.png)](https://i.sstatic.net/2j72n.png)
  
  Show them a ruler with inches on it. Floats can store inch fractions perfectly. This should debunk the myth that floats have rounding errors. Entering these into a computer and doing math on them is a good exercise because it forces the student to do the conversion by hand. These conversions will all happen without discrepancies. You can even challenge the students to try to catch the computer doing inch fraction math wrong.
  
  I used to teach this with drafting rulers. You can get one that has decimal inches. Put the decimal inch ruler next to the fraction inch ruler. Show them how some numbers match up but others don't. The decimal inch ruler represents what you see and what you type. The fraction inch ruler represents what is stored.
  
  [![enter image description here](https://i.sstatic.net/R146f.png)](https://i.sstatic.net/R146f.png)
  
  You can even get students to reproduce the same kinds of errors by having them take measurements of values from the decimal ruler on the fraction ruler and trying to convert back. They'll quickly notice that some convert back easily while others don't. Well, as long as you don't let them simply remember how the story started.
  
  You can find links to other resources in my earlier posts [here](https://softwareengineering.stackexchange.com/a/344327/131624).
  
  > **Comments:**
  > **username_20**: I like how you stress the difference between rounding and conversion. However, your use of terminology is slightly incorrect compared to IEEE754. Conversion is the process of switching between decimal and float. In some cases, switching back and forth will not lose information. However, there is usually still rounding taking place in those conversions, possibly even if after converting back you end up with the same result (the two roundings simply cancelled out in that case). I would be hesitant to oversimplify things here, as that might do more harm than good in the long run.  
  > **username_19**: @username_20 I'm trying to get away from the myth that the discrepancies in floats are just rounding errors.  I've seen to many students turn in code that attempts to correct the problem with their own rounding code. That's the oversimplification to avoid.  
  > **username_21**: The decimal inch ruler is perfect here.  
  
  ---
  
  ### Answer by username_2 on 2017-06-26. Score: 4
  
  Here are a few more ideas:
  
  Point out that that only $2^{32}$ values can be represented in 32 bits (or $2^{64}$ values in 64 bits), while there are an infinite number of real numbers. Therefore, not all real numbers can be represented exactly. For example, if you take the smallest positive number that can be represented exactly and divide it by 2, you can't represent it exactly (by definition).
  
  Explain that floating-point numbers are represented as the sum of powers of 2. Have them compute relatively easy values, such as $1\frac{3}{4} = 1 + \frac{1}{2} + \frac{1}{4}$. Ask them to represent $\frac{1}{3}$ as the sum of powers of 2. They will not be able to.
  
  
  ---
  
  ### Answer by username_8 on 2017-06-26. Score: 2
  
  As a student myself, I had a course about how processing and encoding works where we had a detailed part about operations in different bases and how `floats` and `doubles` differ from one another by the number of bytes allocated, and how the computer's different ways of encoding floating numbers can be done by hand to get a grasp of it, but this wasn't until my third year, after having a first introduction to operations in different bases during my second year.
  
  Only basic understanding of numbers in different bases and knowledge of the fact that `floats` are coded on a given number of bits is necessary to understand why it may be a problem, you could use the example of scientific calculations, where precision is important to make students get the stakes. But if you want students to really understand it fully, I think training operations in different bases and making them try by hand floating numbers coding and rounding may be necessary to get how frequent approximations are and how trivial calculations can give an answer different than what's expected, but more time consuming.
  
  Also, if you want a playful and yet well explained way to teach students how those imprecisions work and how it may affect programs, here is a 10 min long video about floating numbers imprecision in mario 64 that's really well explained : <https://www.youtube.com/watch?v=9hdFG2GcNuA>
  
  
  ---
  
  ### Answer by username_9 on 2021-12-27. Score: 1
  
  There are plenty of good answers here, so I'll just add the extremely simple example I concocted to show why addition is not associative.
  
  Suppose your have a decimal floating point system with one significant digit. Now what is `4+6+7`? If you evaluate left to right it's `2 * 10^1`, if you evaluate right to left it's `1 * 10^1`. (This is with rounding. I give my students a bonus exercise of coming up with an example that uses truncation.)
  
  Worked out: [https://pages.tacc.utexas.edu/~username_9/istc/html/arithmetic.html#Associativity](https://pages.tacc.utexas.edu/%7Eeijkhout/istc/html/arithmetic.html#Associativity) and read the whole page for anything else you want to know about floating point math.
  
  
  ---
  
  