Answers.Body: <p>Slope is not defined like this. You are confusing slope with angle. The slope definition will be more natural, as seen below.</p>
  <p>Intuitively, the slope of a curve at a point is directly how steeply it changes as a function of change of horizontal distance: a flat road doesn't change in height as you drive on it, but driving on a ramp constantly elevates you as you drive up it. Typically this is understood as a ratio, so a 45 degree angle has a slope of 1, and a slope of 30 degrees has a slope of <span class="math-container">$\sqrt3$</span>. This will be formalized in trigonometry, but if I recall correctly, the definition can go the other direction as well (that the ratio approach defines the angle).</p>
  <p>We use ratios measurements of slopes instead of degree measurements of angles because calculating the derivative using the limit definition gives the slope as a ratio directly. <a href="https://en.wikipedia.org/wiki/File:Tangent_animation.gif" rel="nofollow noreferrer">This graphic</a> shows the idea rather nicely, but <a href="https://en.wikipedia.org/wiki/Derivative#Toward_a_definition" rel="nofollow noreferrer">this article</a> provides a clearer exposition.</p>
  <p>There's also the case that the slope as a ratio makes more sense since the backpropagation technique is effectively adding/subtracting <span class="math-container">$ε$</span> to some <span class="math-container">$x$</span> given known <span class="math-container">$\Delta f(x)$</span>, the error signal, relying on the equation you posted above: <span class="math-container">$f(+)≈()+′()$</span>.</p>
  <p>We can rewrite the equation above to obtain <span class="math-container">$() - f(+) ≈ - ′() $</span>. Since <span class="math-container">$\Delta f(x)$</span> and ′() are known, we can also compute <span class="math-container">$$</span>, but note that only the righthand expression of <span class="math-container">$$</span> is computed since <span class="math-container">$() - f(+)$</span> is obtained as <span class="math-container">$Δ()$</span>. We can resolve this with the update rule <span class="math-container">$x \rightarrow x - \epsilon$</span></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-08-24T05:43:50.447Z
Answers.Text: Do you mean $\dfrac{dy}{dx} = \tan \theta$ may not be valid in some cases?
Answers.UserDisplayName: hanugm
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-08-24T05:47:49.683Z
Answers.Text: i think you are confusing the word "tangent" for the trig expression $\tan$. these are different.
Answers.UserDisplayName: k.c. sayz 'k.c sayz'
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-08-24T05:50:00.083Z
Answers.Text: $\theta$ inside $\tan \theta$ is the angle made by which line? It is tangent line right?
Answers.UserDisplayName: hanugm
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-08-24T05:51:56.570Z
Answers.Text: relative to what other angle? you'll eventually have to use the limit approach anyways i believe...
Answers.UserDisplayName: k.c. sayz 'k.c sayz'
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-08-24T05:54:59.200Z
Answers.Text: sorry i meant to say: the line that defines the tangent is characterized by the limit approach; without it, you will find it hard to "draw a line" that only "touches the curve" at a single point
Answers.UserDisplayName: k.c. sayz 'k.c sayz'
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-08-24T05:22:48.470Z
Answers.LastActivityDate: 2021-08-24T05:42:15.100Z
Answers.LastEditDate: 2021-08-24T05:42:15.100Z
Answers.UserDisplayName: k.c. sayz 'k.c sayz'
Body: <p>Consider the following paragraph from <a href="https://www.deeplearningbook.org/contents/numerical.html" rel="nofollow noreferrer">Numerical Computation</a> of deep learning book that says derivative as a slope of the function curve at a point</p>
  <blockquote>
  <p>Suppose we have a function <span class="math-container">$y= f(x)$</span>, where both <span class="math-container">$x$</span> and <span class="math-container">$y$</span> are real
  numbers. The derivative of this function is denoted as <span class="math-container">$f'(x)$</span> or as
  <span class="math-container">$\dfrac{dy}{dx}$</span> . <strong>The derivative <span class="math-container">$f'(x)$</span> gives the slope of <span class="math-container">$f(x)$</span>
  at the point <span class="math-container">$x$</span></strong>. In other words, it speciﬁes how to scale a small
  change in the input to obtain the corresponding change in the output:
  <span class="math-container">$f(x+ \epsilon) \approx f(x)+\epsilon f'(x)$</span>.</p>
  </blockquote>
  <p>Slope of a function <span class="math-container">$f(x)$</span> at a point <span class="math-container">$a$</span> is generally defined as the <span class="math-container">$\tan$</span> of the angle made by the tangent line at the point <span class="math-container">$a$</span> on the curve of the function <span class="math-container">$f(x)$</span> to the positive x-axis in anti-clock wise direction. That is, if <span class="math-container">$\theta$</span> is the angle made by the tangent of the curve <span class="math-container">$f(x)$</span> at a point <span class="math-container">$(a, f(a))$</span> to the positive x-axis in anti-clock wise direction. Then the slope of <span class="math-container">$f(x)$</span> at point <span class="math-container">$a$</span> is <span class="math-container">$\tan \theta$</span>.</p>
  <p>In theory, tangent line should touch the curve of <span class="math-container">$f(x)$</span> at a <a href="https://clas.sa.ucsb.edu/staff/lee/secant,%20tangent,%20and%20derivatives.htm" rel="nofollow noreferrer">single point only</a>. <strong>Most of the textbooks draws <a href="https://clas.sa.ucsb.edu/staff/lee/Secant%20and%20Tangent%20lines.gif" rel="nofollow noreferrer">nice convex curves</a> and then show slope as <span class="math-container">$\tan \theta$</span>.</strong> But, i think it is not possible for many functions to draw a tangent line at a point that touches the curve at that single point only. Else it may be a tangent line or some other traversal.</p>
  <p>How to understand slope as <span class="math-container">$\tan \theta$</span> in such cases? Where am I going wrong?</p>
  
Tags: <math><derivative>
Title: How to understand slope of a (non-convex) function at a point in domain?
UserDisplayName: hanugm
ThreadText: # How to understand slope of a (non-convex) function at a point in domain?
  
  (Asked by: username_0 on 2021-08-24)
  
  Consider the following paragraph from [Numerical Computation](https://www.deeplearningbook.org/contents/numerical.html) of deep learning book that says derivative as a slope of the function curve at a point
  
  > Suppose we have a function $y= f(x)$, where both $x$ and $y$ are real
  > numbers. The derivative of this function is denoted as $f'(x)$ or as
  > $\dfrac{dy}{dx}$ . **The derivative $f'(x)$ gives the slope of $f(x)$
  > at the point $x$**. In other words, it speciﬁes how to scale a small
  > change in the input to obtain the corresponding change in the output:
  > $f(x+ \epsilon) \approx f(x)+\epsilon f'(x)$.
  
  Slope of a function $f(x)$ at a point $a$ is generally defined as the $\tan$ of the angle made by the tangent line at the point $a$ on the curve of the function $f(x)$ to the positive x-axis in anti-clock wise direction. That is, if $\theta$ is the angle made by the tangent of the curve $f(x)$ at a point $(a, f(a))$ to the positive x-axis in anti-clock wise direction. Then the slope of $f(x)$ at point $a$ is $\tan \theta$.
  
  In theory, tangent line should touch the curve of $f(x)$ at a [single point only](https://clas.sa.ucsb.edu/staff/lee/secant,%20tangent,%20and%20derivatives.htm). **Most of the textbooks draws [nice convex curves](https://clas.sa.ucsb.edu/staff/lee/Secant%20and%20Tangent%20lines.gif) and then show slope as $\tan \theta$.** But, i think it is not possible for many functions to draw a tangent line at a point that touches the curve at that single point only. Else it may be a tangent line or some other traversal.
  
  How to understand slope as $\tan \theta$ in such cases? Where am I going wrong?
  
  **Tags:** math, derivative
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-08-24. Score: 0
  
  Slope is not defined like this. You are confusing slope with angle. The slope definition will be more natural, as seen below.
  
  Intuitively, the slope of a curve at a point is directly how steeply it changes as a function of change of horizontal distance: a flat road doesn't change in height as you drive on it, but driving on a ramp constantly elevates you as you drive up it. Typically this is understood as a ratio, so a 45 degree angle has a slope of 1, and a slope of 30 degrees has a slope of $\sqrt3$. This will be formalized in trigonometry, but if I recall correctly, the definition can go the other direction as well (that the ratio approach defines the angle).
  
  We use ratios measurements of slopes instead of degree measurements of angles because calculating the derivative using the limit definition gives the slope as a ratio directly. [This graphic](https://en.wikipedia.org/wiki/File:Tangent_animation.gif) shows the idea rather nicely, but [this article](https://en.wikipedia.org/wiki/Derivative#Toward_a_definition) provides a clearer exposition.
  
  There's also the case that the slope as a ratio makes more sense since the backpropagation technique is effectively adding/subtracting $ε$ to some $x$ given known $\Delta f(x)$, the error signal, relying on the equation you posted above: $f(+)≈()+′()$.
  
  We can rewrite the equation above to obtain $() - f(+) ≈ - ′() $. Since $\Delta f(x)$ and ′() are known, we can also compute $$, but note that only the righthand expression of $$ is computed since $() - f(+)$ is obtained as $Δ()$. We can resolve this with the update rule $x \rightarrow x - \epsilon$
  
  > **Comments:**
  > **username_0**: Do you mean $\dfrac{dy}{dx} = \tan \theta$ may not be valid in some cases?  
  > **username_1**: i think you are confusing the word "tangent" for the trig expression $\tan$. these are different.  
  > **username_0**: $\theta$ inside $\tan \theta$ is the angle made by which line? It is tangent line right?  
  > **username_1**: relative to what other angle? you'll eventually have to use the limit approach anyways i believe...  
  > **username_1**: sorry i meant to say: the line that defines the tangent is characterized by the limit approach; without it, you will find it hard to "draw a line" that only "touches the curve" at a single point  
  
  ---
  
  