Answers.Body: <p>I think that your missing the fact that <em><strong><span class="math-container">$r$</span></strong></em> is not a random constant, but precisely the minimum distance we want to ensure between each point and the separation hyperplane (called margin). And indeed in the book they use <em><span class="math-container">$r$</span></em> to define the distance from a generic vector <span class="math-container">$a$</span> to the hyperplane as well:</p>
  <p><span class="math-container">$a = a' + r\frac{w}{||w||}$</span></p>
  <p>Since <span class="math-container">$\frac{w}{||w||}$</span> is a normalized vector, we can consider <span class="math-container">$a'$</span> as the  x coordinate of <span class="math-container">$a$</span> in the coordinates system defined by <span class="math-container">$w$</span>, vector orthogonal to the hyperplane and the hyperplane itself, whereas <span class="math-container">$r$</span> represent the y coordinate (see drawing below, apologies for the awful quality). To be able to separate 2 points <span class="math-container">$x_1$</span> and <span class="math-container">$x_2$</span> into 2 different classes, we want them to fall into different sides of the hyperplane. By imposing</p>
  <p><span class="math-container">$y(\langle w, a \rangle + b) \geq r$</span></p>
  <p>we're also saying that we don't want the points to just fall into different sides of the hyperplane, but they're y coordinate should also be bigger or equal to r (in absolute value).</p>
  <p>So, to answer your question, <span class="math-container">$(\langle a x \rangle + b) = ||x - x'||$</span> is not really a correct statement but for a bunch of points instead will hold  <span class="math-container">$y(\langle w, a \rangle + b) = x - x'$</span> (i.e. the y coordinate of the point we want to classify express in the coordinates system of the hyperplane is equal to r, which means the point lies in the margin of the hyperplane).</p>
  <p><a href="https://i.sstatic.net/8cewP.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/8cewP.png" alt="enter image description here" /></a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-01-25T20:52:50.020Z
Answers.LastActivityDate: 2022-01-25T20:52:50.020Z
Answers.UserDisplayName: Edoardo Guerriero
Body: <p>Currently, I am trying to understand the mathematics of SVM's using the textbook 'Mathematics for Machine Learning' by Deisenrot et. al.</p>
  <p>On page 375, they define the distance between a point <span class="math-container">$x$</span> and the separating hyperplane as the length of the distance vector <span class="math-container">$||x - x'||$</span>, where <span class="math-container">$x'$</span> corresponds to the orthogonal projection of x onto the hyperplane.</p>
  <p>Subsequently, they formulate the following condition to express that any point's distance from the hyperplane should be greater than or equal to some constant r:</p>
  <p><span class="math-container">$$y(\langle w, x \rangle + b) \geq r$$</span></p>
  <ul>
  <li><span class="math-container">$y ∈ {0, 1}$</span> representing the class of <span class="math-container">$x$</span></li>
  <li><span class="math-container">$w$</span> normal vector and <span class="math-container">$b$</span> the bias.</li>
  </ul>
  <p>So my question is: Does <span class="math-container">$(\langle w, x \rangle + b) = ||x - x'||$</span> hold?</p>
  <p>If that's the case, I would be thankful for some hints on how both sides of the equation are related.</p>
  
Tags: <support-vector-machine><linear-algebra>
Title: Does $(\langle w, x \rangle + b) = ||x - x'||$ hold?
UserDisplayName: Philipp Kunz
ThreadText: # Does $(\langle w, x \rangle + b) = ||x - x'||$ hold?
  
  (Asked by: username_0 on 2022-01-21)
  
  Currently, I am trying to understand the mathematics of SVM's using the textbook 'Mathematics for Machine Learning' by Deisenrot et. al.
  
  On page 375, they define the distance between a point $x$ and the separating hyperplane as the length of the distance vector $||x - x'||$, where $x'$ corresponds to the orthogonal projection of x onto the hyperplane.
  
  Subsequently, they formulate the following condition to express that any point's distance from the hyperplane should be greater than or equal to some constant r:
  
  $$y(\langle w, x \rangle + b) \geq r$$
  
  * $y ∈ {0, 1}$ representing the class of $x$
  * $w$ normal vector and $b$ the bias.
  
  So my question is: Does $(\langle w, x \rangle + b) = ||x - x'||$ hold?
  
  If that's the case, I would be thankful for some hints on how both sides of the equation are related.
  
  **Tags:** support-vector-machine, linear-algebra
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-01-25. Score: 1
  
  I think that your missing the fact that ***$r$*** is not a random constant, but precisely the minimum distance we want to ensure between each point and the separation hyperplane (called margin). And indeed in the book they use *$r$* to define the distance from a generic vector $a$ to the hyperplane as well:
  
  $a = a' + r\frac{w}{||w||}$
  
  Since $\frac{w}{||w||}$ is a normalized vector, we can consider $a'$ as the x coordinate of $a$ in the coordinates system defined by $w$, vector orthogonal to the hyperplane and the hyperplane itself, whereas $r$ represent the y coordinate (see drawing below, apologies for the awful quality). To be able to separate 2 points $x\_1$ and $x\_2$ into 2 different classes, we want them to fall into different sides of the hyperplane. By imposing
  
  $y(\langle w, a \rangle + b) \geq r$
  
  we're also saying that we don't want the points to just fall into different sides of the hyperplane, but they're y coordinate should also be bigger or equal to r (in absolute value).
  
  So, to answer your question, $(\langle a x \rangle + b) = ||x - x'||$ is not really a correct statement but for a bunch of points instead will hold $y(\langle w, a \rangle + b) = x - x'$ (i.e. the y coordinate of the point we want to classify express in the coordinates system of the hyperplane is equal to r, which means the point lies in the margin of the hyperplane).
  
  [![enter image description here](https://i.sstatic.net/8cewP.png)](https://i.sstatic.net/8cewP.png)
  
  
  ---
  
  