Answers.Body: <p>That is the <em><a href="https://en.wikipedia.org/wiki/Hinge_loss" rel="nofollow noreferrer">hinge loss</a></em>, a type of loss most notably used for SVM classification. The hinge loss is typically defined as:</p>
  <p><span class="math-container">$$
  \ell(y)=\max (0,1-t \cdot y),
  $$</span></p>
  <p>which, in your use case, is something like this:</p>
  <p><span class="math-container">$$
  \operatorname{cost}\left(h_{\theta}(x), y\right)=\left\{\begin{array}{ll}
  \max \left(0,1-\theta^{T} x\right) &amp; \text { if } y=1 \\
  \max \left(0,1+\theta^{T} x\right) &amp; \text { if } y=0
  \end{array}\right.
  $$</span></p>
  <p>Check <a href="https://towardsdatascience.com/optimization-loss-function-under-the-hood-part-iii-5dff33fa015d" rel="nofollow noreferrer">this article</a> on it.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-02T13:09:35.713Z
Answers.LastActivityDate: 2020-11-01T22:20:26.683Z
Answers.LastEditDate: 2020-11-01T22:20:26.683Z
Answers.UserDisplayName: JVGD
Body: <p>In a course that I am attending, the cost function of a support vector machine is given by</p>
  <p><span class="math-container">$$J(\theta)=\sum_{i=1}^{m} y^{(i)} \operatorname{cost}_{1}\left(\theta^{T} x^{(i)}\right)+\left(1-y^{(i)}\right) \operatorname{cost}_{0}\left(\theta^{T} x^{(i)}\right)+\frac{\lambda}{2} \sum_{j=1}^{n} \Theta_{j}^{2}$$</span></p>
  <p>where <span class="math-container">$\operatorname{cost}_{1}$</span> and <span class="math-container">$\operatorname{cost}_{0}$</span> look like this (in Magenta):</p>
  <p><a href="https://i.sstatic.net/z2xaT.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/z2xaT.png" alt="enter image description here" /></a></p>
  <p><a href="https://i.sstatic.net/QSIqh.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/QSIqh.png" alt="enter image description here" /></a></p>
  <p>What are the values of the functions <span class="math-container">$\operatorname{cost}_{1}$</span> and <span class="math-container">$\operatorname{cost}_{0}$</span>?</p>
  <p>For example, if using logistic regression the values of <span class="math-container">$\operatorname{cost}_{1}$</span> and <span class="math-container">$\operatorname{cost}_{0}$</span> would be <span class="math-container">$-\log* \operatorname{sigmoid}(-z)$</span> and <span class="math-container">$-\log*(1-\operatorname{sigmoid}(-z))$</span>.</p>
  
Tags: <objective-functions><math><support-vector-machine><hinge-loss>
Title: What is the definition of the "cost" function in the SVM's objective function?
UserDisplayName: jr123456jr987654321
ThreadText: # What is the definition of the "cost" function in the SVM's objective function?
  
  (Asked by: username_0 on 2020-07-02)
  
  In a course that I am attending, the cost function of a support vector machine is given by
  
  $$J(\theta)=\sum\_{i=1}^{m} y^{(i)} \operatorname{cost}\_{1}\left(\theta^{T} x^{(i)}\right)+\left(1-y^{(i)}\right) \operatorname{cost}\_{0}\left(\theta^{T} x^{(i)}\right)+\frac{\lambda}{2} \sum\_{j=1}^{n} \Theta\_{j}^{2}$$
  
  where $\operatorname{cost}\_{1}$ and $\operatorname{cost}\_{0}$ look like this (in Magenta):
  
  [![enter image description here](https://i.sstatic.net/z2xaT.png)](https://i.sstatic.net/z2xaT.png)
  
  [![enter image description here](https://i.sstatic.net/QSIqh.png)](https://i.sstatic.net/QSIqh.png)
  
  What are the values of the functions $\operatorname{cost}\_{1}$ and $\operatorname{cost}\_{0}$?
  
  For example, if using logistic regression the values of $\operatorname{cost}\_{1}$ and $\operatorname{cost}\_{0}$ would be $-\log\* \operatorname{sigmoid}(-z)$ and $-\log\*(1-\operatorname{sigmoid}(-z))$.
  
  **Tags:** objective-functions, math, support-vector-machine, hinge-loss
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-07-02. Score: 2 | Accepted answer
  
  That is the *[hinge loss](https://en.wikipedia.org/wiki/Hinge_loss)*, a type of loss most notably used for SVM classification. The hinge loss is typically defined as:
  
  $$
  \ell(y)=\max (0,1-t \cdot y),
  $$
  
  which, in your use case, is something like this:
  
  $$
  \operatorname{cost}\left(h\_{\theta}(x), y\right)=\left\{\begin{array}{ll}
  \max \left(0,1-\theta^{T} x\right) & \text { if } y=1 \\
  \max \left(0,1+\theta^{T} x\right) & \text { if } y=0
  \end{array}\right.
  $$
  
  Check [this article](https://towardsdatascience.com/optimization-loss-function-under-the-hood-part-iii-5dff33fa015d) on it.
  
  
  ---
  
  