Answers.Body: <p>Since the data is linearly separable linear model <span class="math-container">$y = w^Tx$</span> will be able to perfectly classify all the examples. That means that loss functions <span class="math-container">$L_1(w), L_3(w)$</span> and <span class="math-container">$L_4(w)$</span> will have a value of 0 (since all examples are correctly classified). For the loss <span class="math-container">$L_2(w)$</span> second term will be 0 if all examples are correctly classified. The first term of <span class="math-container">$L_2(w)$</span>
  <span class="math-container">\begin{equation}
  \frac{1}{2}||w||^2_2
  \end{equation}</span>
  will be not be minimized to 0 (unless optimal <span class="math-container">$w = \mathbf{0}$</span>) so <span class="math-container">$L_2(w) &gt; 0$</span>. Optimizer will try to minimize <span class="math-container">$L_2(w)$</span> to be 0, so because of the penalty to <span class="math-container">$w$</span> it will try to find the best tradeoff between minimizing <span class="math-container">$w$</span> and loss due to misclassification so the resulting <span class="math-container">$w$</span> may not be the best to achieve optimal classification.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-08T21:51:07.697Z
Answers.LastActivityDate: 2021-01-08T21:57:23.113Z
Answers.LastEditDate: 2021-01-08T21:57:23.113Z
Answers.UserDisplayName: Brale
Body: <p>I'm studying machine learning and I came into a challenging question.</p>
  <p><a href="https://i.sstatic.net/Tprev.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Tprev.png" alt="enter image description here" /></a></p>
  <p>The answer is 2. But based on my ML notes, all of them are true. Where are the wrong points?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-08T11:30:29.160Z
Comments.Text: Can you please tell us in which context does this question/problem pop up? Is this a homework question? Can you provide the link to the exam/homework/challenge? This seems to be related to support vector machines, but I suggest that you edit your post to provide more context. Moreover, what do your notes actually tell? Do you understand the question? Because I'm not fully sure I really understand this question or, specificallt, what they mean by "less optimum answer".
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-08T11:49:23.807Z
Comments.Text: @nbro the question is about less optimal hyperplanes. if we know the w vector and bias term, we can draw the discriminant hyperplane. As pointed in the question,  the training data are linearly separable. So we may have a lot of discriminant hyperplane. In 2 there is a regularization term, what is its role? It makes the loss function to find less optimal w vectors?
Comments.UserDisplayName: Emiliiii
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-08T11:51:52.200Z
Comments.Text: This really seems to be related to SVMs. Please, provide a link to the resource where you're reading this to have more context. So, as far as I recall, equation 2 is an equation that allows for some tolerance and that some points may not be correctly classified. Check the soft-margin here: https://en.wikipedia.org/wiki/Support-vector_machine for a similar formula.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-08T12:03:40.097Z
Comments.Text: Would you clarify a bit more? what will happen when C goes to infinity?
Comments.UserDisplayName: Emiliiii
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-08T12:17:34.123Z
Comments.Text: @DuttaA what is the difference between option 2 and others?
Comments.UserDisplayName: Emiliiii
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-08T13:21:14.813Z
Comments.Text: @DuttaA I want to know is there any difference between when C goes to infinity and when it has a good value.
Comments.UserDisplayName: Emiliiii
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-08T14:26:58.337Z
Comments.Text: @DuttaA I mean the interpretation in view of machine learning  when C goes to infinity and when it doesn't, not mathematically
Comments.UserDisplayName: Emiliiii
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-08T18:34:15.143Z
Comments.Text: @DuttaA there is no deadline ever. this is very challenging question. I'm wait forever to learn new things. thanks
Comments.UserDisplayName: Emiliiii
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-08T19:24:04.163Z
Comments.Text: @Emiliiii As I asked before, if you provide the link to the context/website/whatever lists this challenge, then maybe we can have more context to answer it properly.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-08T22:01:56.063Z
Comments.Text: @DuttaA The question doesn't mention about bias term.
Comments.UserDisplayName: Emiliiii
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-08T22:04:16.877Z
Comments.Text: Sorry @nbro I didn't see your comment. there is no link here. this is hard-copy material gives us by TA.
Comments.UserDisplayName: Emiliiii
Tags: <machine-learning><classification><objective-functions><support-vector-machine><homework>
Title: If the training data are linearly separable, which of the following $L(w)$ has less optimum answer for $w$, when $y = w^Tx$?
UserDisplayName: Emiliiii
ThreadText: # If the training data are linearly separable, which of the following $L(w)$ has less optimum answer for $w$, when $y = w^Tx$?
  
  (Asked by: username_0 on 2021-01-08)
  
  I'm studying machine learning and I came into a challenging question.
  
  [![enter image description here](https://i.sstatic.net/Tprev.png)](https://i.sstatic.net/Tprev.png)
  
  The answer is 2. But based on my ML notes, all of them are true. Where are the wrong points?
  
  **Tags:** machine-learning, classification, objective-functions, support-vector-machine, homework
  > **Comments:**
  > **username_1**: Can you please tell us in which context does this question/problem pop up? Is this a homework question? Can you provide the link to the exam/homework/challenge? This seems to be related to support vector machines, but I suggest that you edit your post to provide more context. Moreover, what do your notes actually tell? Do you understand the question? Because I'm not fully sure I really understand this question or, specificallt, what they mean by "less optimum answer".  
  > **username_0**: @nbro the question is about less optimal hyperplanes. if we know the w vector and bias term, we can draw the discriminant hyperplane. As pointed in the question,  the training data are linearly separable. So we may have a lot of discriminant hyperplane. In 2 there is a regularization term, what is its role? It makes the loss function to find less optimal w vectors?  
  > **username_1**: This really seems to be related to SVMs. Please, provide a link to the resource where you're reading this to have more context. So, as far as I recall, equation 2 is an equation that allows for some tolerance and that some points may not be correctly classified. Check the soft-margin here: https://en.wikipedia.org/wiki/Support-vector_machine for a similar formula.  
  > **username_0**: Would you clarify a bit more? what will happen when C goes to infinity?  
  > **username_0**: @DuttaA what is the difference between option 2 and others?  
  > **username_0**: @DuttaA I want to know is there any difference between when C goes to infinity and when it has a good value.  
  > **username_0**: @DuttaA I mean the interpretation in view of machine learning  when C goes to infinity and when it doesn't, not mathematically  
  > **username_0**: @DuttaA there is no deadline ever. this is very challenging question. I'm wait forever to learn new things. thanks  
  > **username_1**: @Emiliiii As I asked before, if you provide the link to the context/website/whatever lists this challenge, then maybe we can have more context to answer it properly.  
  > **username_0**: @DuttaA The question doesn't mention about bias term.  
  > **username_0**: Sorry @nbro I didn't see your comment. there is no link here. this is hard-copy material gives us by TA.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2021-01-08. Score: 0
  
  Since the data is linearly separable linear model $y = w^Tx$ will be able to perfectly classify all the examples. That means that loss functions $L\_1(w), L\_3(w)$ and $L\_4(w)$ will have a value of 0 (since all examples are correctly classified). For the loss $L\_2(w)$ second term will be 0 if all examples are correctly classified. The first term of $L\_2(w)$
  \begin{equation}
  \frac{1}{2}||w||^2\_2
  \end{equation}
  will be not be minimized to 0 (unless optimal $w = \mathbf{0}$) so $L\_2(w) > 0$. Optimizer will try to minimize $L\_2(w)$ to be 0, so because of the penalty to $w$ it will try to find the best tradeoff between minimizing $w$ and loss due to misclassification so the resulting $w$ may not be the best to achieve optimal classification.
  
  
  ---
  
  