Answers.Body: <p>Support vector machines are supervised learning models with associated learning algorithms that analyze data and are used for classification and regression analysis. </p>
  
  <p>Here is a link where you can learn more about it from a introduction level: <a href="https://towardsdatascience.com/support-vector-machine-introduction-to-machine-learning-algorithms-934a444fca47" rel="nofollow noreferrer">"Support Vector Machine — Introduction to Machine Learning Algorithms" (Medium)</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-16T01:41:16.567Z
Answers.LastActivityDate: 2019-08-16T20:25:33.543Z
Answers.LastEditDate: 2019-08-16T20:25:33.543Z
Answers.UserDisplayName: 111
Answers.Body: <p>In case this is still relevant to you I can share my tutorial on SVM with Python implementation in Jupyter notebook:</p>
  
  <p><a href="https://github.com/jjepsuomi/Primer-to-support-vector-machines/blob/master/Primer_to_SVM.ipynb" rel="nofollow noreferrer">Primer to support vector machines</a></p>
  
  <p>The tutorial assumes some mathematics and programming background knowledge. The SVM codes utilize no external machine learning packages and tries to teach the reader to build a SVM model him-/herself.</p>
  
  <p>I hope it helps you!</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-15T12:06:21.860Z
Answers.LastActivityDate: 2019-08-15T12:06:21.860Z
Answers.UserDisplayName: jjepsuomi
Answers.Body: <p>I find the chapter on machine learning from <a href="https://rads.stackoverflow.com/amzn/click/0136042597" rel="noreferrer">Russell &amp; Norvig</a> is a pretty good place to start with SVMs. I think this is Chapter 18?</p>
  
  <p>One way to understand an SVM is as a kind of neural network, but this is not usually an intuitive approach for a beginner (unless your NN knowledge is already quite good).</p>
  
  <p>A better way to understand SVMs is as consisting of three simple ideas rolled into one algorithm. Here's an attempt at a "For Dummies" answer though:</p>
  
  <ol>
  <li><p><strong>Maximum Margin Classification.</strong> SVMs are usually used to find a pattern in a set of data. Often, the data allow an infinite set of possible patterns that are all equally descriptive. For example, maybe The relationship is "Lives within 5 miles of a Coast -> Income High". It's easy to imagine that this pattern is just as good as "Lives within 5.0001 miles of a Coast -> Income High" or "Lives within 4.999 miles of a Coast -> Income High". There might actually be a lot more play than that in the data (e.g. 3 miles might work out too). If all these are equally good, then the maximum margin idea says you should pick the one that's "in the middle" of the data. So maybe all values between 5.5 and 4.8 are equally good. In that case, we might pick 5.15 (in the middle). This example is super simplified. Real world data would have a lot more variables, and the idea of "in the middle" ends up being a little more complex, but this is the intuition. It turns out that finding the maximum margin pattern is easy when the patterns are <em>linear</em>. That is, when they can be represented by drawing straight lines through a plot of the dataset.</p></li>
  <li><p><strong>Projection into higher dimensions.</strong> This one needs a bit of math to visualize. Consider a dataset consisting of a circular pattern (for instance, maybe the pattern is that higher incomes are found in the middle of the city). There is no <em>linear</em> relationship that captures this pattern. That is, you can't draw a straight line through the data, and say something meaningful about all the values on one side or the other. However, if you add a new feature to your data that is the square of the original coordinates, it's easy to find such a pattern. Basically, if you pre-compute "circular" functions of the original data, you can add them to the dataset, and then find a pattern that is a linear function of this new feature. This idea generalizes: if you compute a complex enough function of your original data, and then apply the maximum margin idea, you can learn any pattern you like. The problem is that it's slow: adding more features makes it take longer to find the patterns you want.</p></li>
  <li><p><strong>The Kernel Trick</strong>. The thing that made SVMs useful was the kernel trick: finding the maximum margin didn't depend on anything except the product of the coordinates of the various points. It turned out that this product could be computed first, and then run through certain functions to produce a problem that was <em>identical</em> to the one you'd get by first adding extra features and then doing the multiplication. However, computing the problem this way didn't require adding any new features! This made SVMs one of the first reliable, well understood, and fast methods for finding non-linear patterns in data. </p></li>
  </ol>
  
  <p>Hope that provides a starting point. Consider reading Russell &amp; Norvig as a next starting point, or <a href="https://rads.stackoverflow.com/amzn/click/0387310738" rel="noreferrer">Bishop</a> if you want to go deeper. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-08-03T12:45:17.597Z
Answers.Text: Excellent explanation! So, from my understanding of (2), SVMs do not only classify data that are separable by a single line but they classify also data that could be inside/outside of a circle (any kind of pattern)? Is their output a kind of a binary classification (meaning that the data belongs to a specific class or not) or it can classify the data in more than 2 (3, 4, 5, etc.) categories? eg. "Lives within 1mile of coast -> Income High, lives from 1 to 2 miles from coast -> Income medium, lives to more than 2 miles of coast -> Income Low).
Answers.UserDisplayName: Efthymios Kalyviotis
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-08-03T13:33:21.920Z
Answers.Text: Sort of. They're a combination of all three ideas. So the key is that they can find classification patterns that are non-linear (any kind of pattern), but they do this by turning that problem into the problem of finding a line in a different space. Regular SVMs do binary classification only, but there are lots of variations, some of which do multi-label or multi-class classification. You can also use something like 1vA classification to achieve the same thing (see Rifkin & Klautau: http://www.jmlr.org/papers/volume5/rifkin04a/rifkin04a.pdf).
Answers.UserDisplayName: John Doucette
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-08-02T21:03:46.043Z
Answers.LastActivityDate: 2018-08-03T13:35:46.950Z
Answers.LastEditDate: 2018-08-03T13:35:46.950Z
Answers.UserDisplayName: John Doucette
Body: <p>What is a support vector machine (SVM)? Is an SVM a kind of a neural network, meaning it has nodes and weights, etc.? What is it best used for?</p>
  
  <p>Where I can find information about these?</p>
  
Tags: <neural-networks><machine-learning><definitions><support-vector-machine>
Title: What is a support vector machine?
UserDisplayName: Efthymios Kalyviotis
ThreadText: # What is a support vector machine?
  
  (Asked by: username_0 on 2018-08-02)
  
  What is a support vector machine (SVM)? Is an SVM a kind of a neural network, meaning it has nodes and weights, etc.? What is it best used for?
  
  Where I can find information about these?
  
  **Tags:** neural-networks, machine-learning, definitions, support-vector-machine
  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_3 on 2018-08-02. Score: 6 | Accepted answer
  
  I find the chapter on machine learning from [Russell & Norvig](https://rads.stackoverflow.com/amzn/click/0136042597) is a pretty good place to start with SVMs. I think this is Chapter 18?
  
  One way to understand an SVM is as a kind of neural network, but this is not usually an intuitive approach for a beginner (unless your NN knowledge is already quite good).
  
  A better way to understand SVMs is as consisting of three simple ideas rolled into one algorithm. Here's an attempt at a "For Dummies" answer though:
  
  1. **Maximum Margin Classification.** SVMs are usually used to find a pattern in a set of data. Often, the data allow an infinite set of possible patterns that are all equally descriptive. For example, maybe The relationship is "Lives within 5 miles of a Coast -> Income High". It's easy to imagine that this pattern is just as good as "Lives within 5.0001 miles of a Coast -> Income High" or "Lives within 4.999 miles of a Coast -> Income High". There might actually be a lot more play than that in the data (e.g. 3 miles might work out too). If all these are equally good, then the maximum margin idea says you should pick the one that's "in the middle" of the data. So maybe all values between 5.5 and 4.8 are equally good. In that case, we might pick 5.15 (in the middle). This example is super simplified. Real world data would have a lot more variables, and the idea of "in the middle" ends up being a little more complex, but this is the intuition. It turns out that finding the maximum margin pattern is easy when the patterns are *linear*. That is, when they can be represented by drawing straight lines through a plot of the dataset.
  2. **Projection into higher dimensions.** This one needs a bit of math to visualize. Consider a dataset consisting of a circular pattern (for instance, maybe the pattern is that higher incomes are found in the middle of the city). There is no *linear* relationship that captures this pattern. That is, you can't draw a straight line through the data, and say something meaningful about all the values on one side or the other. However, if you add a new feature to your data that is the square of the original coordinates, it's easy to find such a pattern. Basically, if you pre-compute "circular" functions of the original data, you can add them to the dataset, and then find a pattern that is a linear function of this new feature. This idea generalizes: if you compute a complex enough function of your original data, and then apply the maximum margin idea, you can learn any pattern you like. The problem is that it's slow: adding more features makes it take longer to find the patterns you want.
  3. **The Kernel Trick**. The thing that made SVMs useful was the kernel trick: finding the maximum margin didn't depend on anything except the product of the coordinates of the various points. It turned out that this product could be computed first, and then run through certain functions to produce a problem that was *identical* to the one you'd get by first adding extra features and then doing the multiplication. However, computing the problem this way didn't require adding any new features! This made SVMs one of the first reliable, well understood, and fast methods for finding non-linear patterns in data.
  
  Hope that provides a starting point. Consider reading Russell & Norvig as a next starting point, or [Bishop](https://rads.stackoverflow.com/amzn/click/0387310738) if you want to go deeper.
  
  > **Comments:**
  > **username_0**: Excellent explanation! So, from my understanding of (2), SVMs do not only classify data that are separable by a single line but they classify also data that could be inside/outside of a circle (any kind of pattern)? Is their output a kind of a binary classification (meaning that the data belongs to a specific class or not) or it can classify the data in more than 2 (3, 4, 5, etc.) categories? eg. "Lives within 1mile of coast -> Income High, lives from 1 to 2 miles from coast -> Income medium, lives to more than 2 miles of coast -> Income Low).  
  > **username_3**: Sort of. They're a combination of all three ideas. So the key is that they can find classification patterns that are non-linear (any kind of pattern), but they do this by turning that problem into the problem of finding a line in a different space. Regular SVMs do binary classification only, but there are lots of variations, some of which do multi-label or multi-class classification. You can also use something like 1vA classification to achieve the same thing (see Rifkin & Klautau: http://www.jmlr.org/papers/volume5/rifkin04a/rifkin04a.pdf).  
  
  ---
  
  ### Answer by username_1 on 2019-08-16. Score: 1
  
  Support vector machines are supervised learning models with associated learning algorithms that analyze data and are used for classification and regression analysis.
  
  Here is a link where you can learn more about it from a introduction level: ["Support Vector Machine — Introduction to Machine Learning Algorithms" (Medium)](https://towardsdatascience.com/support-vector-machine-introduction-to-machine-learning-algorithms-934a444fca47)
  
  
  ---
  
  ### Answer by username_2 on 2019-08-15. Score: 1
  
  In case this is still relevant to you I can share my tutorial on SVM with Python implementation in Jupyter notebook:
  
  [Primer to support vector machines](https://github.com/username_2/Primer-to-support-vector-machines/blob/master/Primer_to_SVM.ipynb)
  
  The tutorial assumes some mathematics and programming background knowledge. The SVM codes utilize no external machine learning packages and tries to teach the reader to build a SVM model him-/herself.
  
  I hope it helps you!
  
  
  ---
  
  