Answers.Body: <p>Yes, they can be related to underflow. Mathematically, we do not expect facing with division by zero when we have an expression <span class="math-container">$\frac{1}{\varepsilon}$</span> when <span class="math-container">$\varepsilon &gt; 0$</span>. However, in numerical softwares, it can happen if <span class="math-container">$\varepsilon &lt; lf$</span>.</p>
  <p>Moreover, you can think about the same scenario for taking the logarithm of zero.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-08-17T09:29:11.370Z
Answers.LastActivityDate: 2021-08-17T09:29:11.370Z
Answers.UserDisplayName: OmG
Body: <p>Consider an architecture or programming language that uses <span class="math-container">$n$</span> bits for storing a floating point number in a particular format. Then each and every floating point number it can store should be in a given range, say <span class="math-container">$[lf, uf]$</span>.</p>
  <p>If there is a need to store any floating point number less than <span class="math-container">$lf$</span> then we generally treat such phenomenon as underflow. Consider the following from <a href="https://www.deeplearningbook.org/contents/numerical.html" rel="nofollow noreferrer">Numerical Computation chapter of Deep Learning book</a>.</p>
  <blockquote>
  <p>One form of rounding error that is particularly devastating is
  underﬂow . Underﬂow occurs when numbers near zero are rounded to zero.
  Many functions behave qualitatively diﬀerently when their argument is
  zero rather than a small positive number. For example, we usually want
  to avoid <strong>division by zero</strong> (some software environments will raise
  exceptions when this occurs, others will return a result with a
  placeholder <strong>not-a-number value</strong>) or <strong>taking the logarithm of zero</strong>
  (this is usually treated as <span class="math-container">$-\infty$</span>, which then becomes <strong>not-a-number</strong>
  if it is used for many further arithmetic operations).</p>
  </blockquote>
  <p>You can observe that two examples has been given while explaining underflow: division by zero and logarithm of zero. If we treat mathematically, both are <strong>undefined</strong>. It should not be an issue of storage, especially underflow.</p>
  <p>Is there any reason behind proving such examples, which are mathematically undefined, under the umbrella term <strong>underflow</strong> and using the term &quot;not-a-number&quot;?</p>
  
Tags: <terminology><math><implementation><books><storage>
Title: Why not undefined expression is different from numerical underflow?
UserDisplayName: hanugm
ThreadText: # Why not undefined expression is different from numerical underflow?
  
  (Asked by: username_0 on 2021-08-16)
  
  Consider an architecture or programming language that uses $n$ bits for storing a floating point number in a particular format. Then each and every floating point number it can store should be in a given range, say $[lf, uf]$.
  
  If there is a need to store any floating point number less than $lf$ then we generally treat such phenomenon as underflow. Consider the following from [Numerical Computation chapter of Deep Learning book](https://www.deeplearningbook.org/contents/numerical.html).
  
  > One form of rounding error that is particularly devastating is
  > underﬂow . Underﬂow occurs when numbers near zero are rounded to zero.
  > Many functions behave qualitatively diﬀerently when their argument is
  > zero rather than a small positive number. For example, we usually want
  > to avoid **division by zero** (some software environments will raise
  > exceptions when this occurs, others will return a result with a
  > placeholder **not-a-number value**) or **taking the logarithm of zero**
  > (this is usually treated as $-\infty$, which then becomes **not-a-number**
  > if it is used for many further arithmetic operations).
  
  You can observe that two examples has been given while explaining underflow: division by zero and logarithm of zero. If we treat mathematically, both are **undefined**. It should not be an issue of storage, especially underflow.
  
  Is there any reason behind proving such examples, which are mathematically undefined, under the umbrella term **underflow** and using the term "not-a-number"?
  
  **Tags:** terminology, math, implementation, books, storage
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-08-17. Score: 1
  
  Yes, they can be related to underflow. Mathematically, we do not expect facing with division by zero when we have an expression $\frac{1}{\varepsilon}$ when $\varepsilon > 0$. However, in numerical softwares, it can happen if $\varepsilon < lf$.
  
  Moreover, you can think about the same scenario for taking the logarithm of zero.
  
  
  ---
  
  