<p>That is the <em><a href="https://en.wikipedia.org/wiki/Hinge_loss" rel="nofollow noreferrer">hinge loss</a></em>, a type of loss most notably used for SVM classification. The hinge loss is typically defined as:</p>
<p><span class="math-container">$$
\ell(y)=\max (0,1-t \cdot y),
$$</span></p>
<p>which, in your use case, is something like this:</p>
<p><span class="math-container">$$
\operatorname{cost}\left(h_{\theta}(x), y\right)=\left\{\begin{array}{ll}
\max \left(0,1-\theta^{T} x\right) &amp; \text { if } y=1 \\
\max \left(0,1+\theta^{T} x\right) &amp; \text { if } y=0
\end{array}\right.
$$</span></p>
<p>Check <a href="https://towardsdatascience.com/optimization-loss-function-under-the-hood-part-iii-5dff33fa015d" rel="nofollow noreferrer">this article</a> on it.</p>

CC BY-SA 4.0
2020-07-02T13:09:35.713Z
2020-11-01T22:20:26.683Z
2020-11-01T22:20:26.683Z
JVGD
<p>In a course that I am attending, the cost function of a support vector machine is given by</p>
<p><span class="math-container">$$J(\theta)=\sum_{i=1}^{m} y^{(i)} \operatorname{cost}_{1}\left(\theta^{T} x^{(i)}\right)+\left(1-y^{(i)}\right) \operatorname{cost}_{0}\left(\theta^{T} x^{(i)}\right)+\frac{\lambda}{2} \sum_{j=1}^{n} \Theta_{j}^{2}$$</span></p>
<p>where <span class="math-container">$\operatorname{cost}_{1}$</span> and <span class="math-container">$\operatorname{cost}_{0}$</span> look like this (in Magenta):</p>
<p><a href="https://i.sstatic.net/z2xaT.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/z2xaT.png" alt="enter image description here" /></a></p>
<p><a href="https://i.sstatic.net/QSIqh.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/QSIqh.png" alt="enter image description here" /></a></p>
<p>What are the values of the functions <span class="math-container">$\operatorname{cost}_{1}$</span> and <span class="math-container">$\operatorname{cost}_{0}$</span>?</p>
<p>For example, if using logistic regression the values of <span class="math-container">$\operatorname{cost}_{1}$</span> and <span class="math-container">$\operatorname{cost}_{0}$</span> would be <span class="math-container">$-\log* \operatorname{sigmoid}(-z)$</span> and <span class="math-container">$-\log*(1-\operatorname{sigmoid}(-z))$</span>.</p>

<objective-functions><math><support-vector-machine><hinge-loss>
What is the definition of the "cost" function in the SVM's objective function?
jr123456jr987654321
# What is the definition of the "cost" function in the SVM's objective function?

(Asked by: username_0 on 2020-07-02)

In a course that I am attending, the cost function of a support vector machine is given by

$$J(\theta)=\sum\_{i=1}^{m} y^{(i)} \operatorname{cost}\_{1}\left(\theta^{T} x^{(i)}\right)+\left(1-y^{(i)}\right) \operatorname{cost}\_{0}\left(\theta^{T} x^{(i)}\right)+\frac{\lambda}{2} \sum\_{j=1}^{n} \Theta\_{j}^{2}$$

where $\operatorname{cost}\_{1}$ and $\operatorname{cost}\_{0}$ look like this (in Magenta):

[![enter image description here](https://i.sstatic.net/z2xaT.png)](https://i.sstatic.net/z2xaT.png)

[![enter image description here](https://i.sstatic.net/QSIqh.png)](https://i.sstatic.net/QSIqh.png)

What are the values of the functions $\operatorname{cost}\_{1}$ and $\operatorname{cost}\_{0}$?

For example, if using logistic regression the values of $\operatorname{cost}\_{1}$ and $\operatorname{cost}\_{0}$ would be $-\log\* \operatorname{sigmoid}(-z)$ and $-\log\*(1-\operatorname{sigmoid}(-z))$.

**Tags:** objective-functions, math, support-vector-machine, hinge-loss


---

## 1 answers

### Answer by username_1 on 2020-07-02. Score: 2 | Accepted answer

That is the *[hinge loss](https://en.wikipedia.org/wiki/Hinge_loss)*, a type of loss most notably used for SVM classification. The hinge loss is typically defined as:

$$
\ell(y)=\max (0,1-t \cdot y),
$$

which, in your use case, is something like this:

$$
\operatorname{cost}\left(h\_{\theta}(x), y\right)=\left\{\begin{array}{ll}
\max \left(0,1-\theta^{T} x\right) & \text { if } y=1 \\
\max \left(0,1+\theta^{T} x\right) & \text { if } y=0
\end{array}\right.
$$

Check [this article](https://towardsdatascience.com/optimization-loss-function-under-the-hood-part-iii-5dff33fa015d) on it.


---

