Answers.Body: <p>So actually I managed to get hold of my lecturer to explain the argmax to argmin conversion.</p>
  <p>Generally speaking maximising <span class="math-container">$\frac{1}{||w||}$</span> is identical to minimising <span class="math-container">$||w||$</span>. As <span class="math-container">$||w||$</span> in <span class="math-container">$\frac{1}{||w||}$</span> decreases, the overall value increases, i.e. we maximise it. The reason for choosing <span class="math-container">$\frac{1}{2}||w||^2$</span> turns out to be a less mathematic and more practical one as our optimisation algorithms used perform better on quadratic functions and the <span class="math-container">$\frac{1}{2}$</span> seems to be a rather arbitrary scaling choice.</p>
  <p>If anyone has something to add, I'd love to hear any details though!</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-28T03:28:29.623Z
Answers.LastActivityDate: 2020-10-28T14:53:17.347Z
Answers.LastEditDate: 2020-10-28T14:53:17.347Z
Answers.UserDisplayName: Joneron
Body: <p>I'm new to the AI Stackexchange and wasn't certain if this should go here or to Maths instead but thought the context with ML may be useful to understand my problem. I hope posting this question here could help another student learning about Support Vector Machines some day.</p>
  <p>I'm currently learning about Support Vector Machines at university and came across a weird step I could not understand. We were talking about basic SVMs and formulated the optimisation problem <span class="math-container">$\max_{w,b} \{ \frac{1}{||w||} \min_n(y^{(n)}f(x^{(n)}))\}$</span> which we then simplified down to <span class="math-container">$\max_{w,b} \{ \frac{1}{||w||}\}$</span> by introducing <span class="math-container">$\kappa$</span> as a scaling factor for <span class="math-container">$w$</span> and <span class="math-container">$b$</span> according to the margin of the SVM. Now our lecturer converted it without explanation into a quadratic optimisation problem as <span class="math-container">$\min_{w,b}\{\frac{1}{2} ||w||^2\}$</span> which I could not explain myself. I hope someone with context can help me how this is possible and what math or trick is behind this approach?</p>
  <hr />
  <p>Notation information:</p>
  <ul>
  <li><span class="math-container">$w$</span> - weight matrix</li>
  <li><span class="math-container">$b$</span> - bias (sometimes denoted <span class="math-container">$w_0$</span> I believe?)</li>
  <li><span class="math-container">$x^{(n)}$</span> - Independent variable (vector)</li>
  <li><span class="math-container">$y^{(n)}$</span> - Dependent variable (scalar classifying the input in a binary classifcation as <span class="math-container">$y=1$</span> or <span class="math-container">$y=-1$</span>)</li>
  </ul>
  <p>Thank you very much!</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-10-27T01:33:37.807Z
Comments.Text: Welcome to SE:AI!  I'm glad you asked here as this involves supervised learning.
Comments.UserDisplayName: DukeZhou
Tags: <machine-learning><classification><support-vector-machine><binary-classification>
Title: Support Vector Machine Convert optimisation problem from argmax to argmin
UserDisplayName: Joneron
ThreadText: # Support Vector Machine Convert optimisation problem from argmax to argmin
  
  (Asked by: username_0 on 2020-10-27)
  
  I'm new to the AI Stackexchange and wasn't certain if this should go here or to Maths instead but thought the context with ML may be useful to understand my problem. I hope posting this question here could help another student learning about Support Vector Machines some day.
  
  I'm currently learning about Support Vector Machines at university and came across a weird step I could not understand. We were talking about basic SVMs and formulated the optimisation problem $\max\_{w,b} \{ \frac{1}{||w||} \min\_n(y^{(n)}f(x^{(n)}))\}$ which we then simplified down to $\max\_{w,b} \{ \frac{1}{||w||}\}$ by introducing $\kappa$ as a scaling factor for $w$ and $b$ according to the margin of the SVM. Now our lecturer converted it without explanation into a quadratic optimisation problem as $\min\_{w,b}\{\frac{1}{2} ||w||^2\}$ which I could not explain myself. I hope someone with context can help me how this is possible and what math or trick is behind this approach?
  
  ---
  
  Notation information:
  
  * $w$ - weight matrix
  * $b$ - bias (sometimes denoted $w\_0$ I believe?)
  * $x^{(n)}$ - Independent variable (vector)
  * $y^{(n)}$ - Dependent variable (scalar classifying the input in a binary classifcation as $y=1$ or $y=-1$)
  
  Thank you very much!
  
  **Tags:** machine-learning, classification, support-vector-machine, binary-classification
  > **Comments:**
  > **username_1**: Welcome to SE:AI!  I'm glad you asked here as this involves supervised learning.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2020-10-28. Score: 1
  
  So actually I managed to get hold of my lecturer to explain the argmax to argmin conversion.
  
  Generally speaking maximising $\frac{1}{||w||}$ is identical to minimising $||w||$. As $||w||$ in $\frac{1}{||w||}$ decreases, the overall value increases, i.e. we maximise it. The reason for choosing $\frac{1}{2}||w||^2$ turns out to be a less mathematic and more practical one as our optimisation algorithms used perform better on quadratic functions and the $\frac{1}{2}$ seems to be a rather arbitrary scaling choice.
  
  If anyone has something to add, I'd love to hear any details though!
  
  
  ---
  
  