Answers.Body: <p>I've summarized the <a href="https://martin-thoma.com/svm-with-sklearn/" rel="nofollow noreferrer">key ideas of SVMs</a>. So this is how <span class="math-container">$\gamma$</span> is used with a gaussian Kernel:</p>
  
  <p><span class="math-container">$$K_{\text{Gauss}}(\mathbf{x}_i, \mathbf{x}_j) = e^{\frac{-\gamma\|\mathbf{x}_i - \mathbf{x}_j\|^2}{2 \sigma^2}}$$</span></p>
  
  <p>The bigger the <span class="math-container">$\gamma$</span>, the more "linear" the decision boundary will be. The closer to 0, the more support vectors you have/the more non-linear the boundary is (see <a href="https://cs.stanford.edu/people/karpathy/svmjs/demo/" rel="nofollow noreferrer"><strong>interactive example</strong></a>)</p>
  
  <p>You can also find it on udacity: <a href="https://www.youtube.com/watch?v=m2a2K4lprQw" rel="nofollow noreferrer">SVM Gamma Parameter</a></p>
  
  <p>In practice, you can use a <a href="http://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html#sklearn.model_selection.GridSearchCV" rel="nofollow noreferrer">grid search</a> or random search to get good values.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-01T09:02:00.137Z
Answers.Text: I think you meant:
  the bigger the $\sigma$, or equivalently, the smaller the $\gamma$, the more "linear" the decision boundary will be
Answers.UserDisplayName: Radu
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-09-24T06:00:33.820Z
Answers.LastActivityDate: 2018-09-24T06:00:33.820Z
Answers.UserDisplayName: Martin Thoma
Answers.Body: <p>Rougly speaking, the higher the gamma, the more complex the model, the higher the risk of overfitting.</p>
  
  <p>In fact, as you can read on the page you linked:</p>
  
  <blockquote>
    <p>If <strong>gamma is too large</strong>, the radius of the area of influence of the
    support vectors <strong>only includes the support vector itself</strong>[...]</p>
    
    <p>When <strong>gamma is very small</strong>, the model is too constrained and <strong>cannot
    capture the complexity or “shape” of the data</strong>. The <strong>region of influence
    of any selected support vector would include the whole training set</strong>.</p>
  </blockquote>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-09-23T16:55:05.227Z
Answers.Text: i not understanding this "of a single training example reaches"
Answers.UserDisplayName: information
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-09-23T17:04:58.833Z
Answers.Text: Replace "a single" with "each individual".
Answers.UserDisplayName: sentence
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-09-23T16:36:16.490Z
Answers.LastActivityDate: 2018-09-23T16:36:16.490Z
Answers.UserDisplayName: sentence
Body: <p>I want to understand what the <code>gamma</code> parameter does in an SVM. According to <a href="http://scikit-learn.org/stable/auto_examples/svm/plot_rbf_parameters.html" rel="nofollow noreferrer">this page</a>.</p>
  <blockquote>
  <p>Intuitively, the <code>gamma</code> parameter defines how far the influence of a single training example reaches, with low values meaning ‘far’ and high values meaning ‘close’. The <code>gamma</code> parameters can be seen as the inverse of the radius of influence of samples selected by the model as support vectors.</p>
  </blockquote>
  <p>I don't understand this part &quot;<strong>of a single training example reaches</strong>&quot;, does it refer to the training dataset?</p>
  
Tags: <machine-learning><support-vector-machine><hyper-parameters>
Title: What is the purpose of the "gamma" parameter in SVMs?
UserDisplayName: information
ThreadText: # What is the purpose of the "gamma" parameter in SVMs?
  
  (Asked by: username_0 on 2018-09-23)
  
  I want to understand what the `gamma` parameter does in an SVM. According to [this page](http://scikit-learn.org/stable/auto_examples/svm/plot_rbf_parameters.html).
  
  > Intuitively, the `gamma` parameter defines how far the influence of a single training example reaches, with low values meaning ‘far’ and high values meaning ‘close’. The `gamma` parameters can be seen as the inverse of the radius of influence of samples selected by the model as support vectors.
  
  I don't understand this part "**of a single training example reaches**", does it refer to the training dataset?
  
  **Tags:** machine-learning, support-vector-machine, hyper-parameters
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2018-09-24. Score: 2
  
  I've summarized the [key ideas of SVMs](https://username_1-username_1.com/svm-with-sklearn/). So this is how $\gamma$ is used with a gaussian Kernel:
  
  $$K\_{\text{Gauss}}(\mathbf{x}\_i, \mathbf{x}\_j) = e^{\frac{-\gamma\|\mathbf{x}\_i - \mathbf{x}\_j\|^2}{2 \sigma^2}}$$
  
  The bigger the $\gamma$, the more "linear" the decision boundary will be. The closer to 0, the more support vectors you have/the more non-linear the boundary is (see [**interactive example**](https://cs.stanford.edu/people/karpathy/svmjs/demo/))
  
  You can also find it on udacity: [SVM Gamma Parameter](https://www.youtube.com/watch?v=m2a2K4lprQw)
  
  In practice, you can use a [grid search](http://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html#sklearn.model_selection.GridSearchCV) or random search to get good values.
  
  > **Comments:**
  > **username_2**: I think you meant: the bigger the $\sigma$, or equivalently, the smaller the $\gamma$, the more "linear" the decision boundary will be  
  
  ---
  
  ### Answer by username_3 on 2018-09-23. Score: 1
  
  Rougly speaking, the higher the gamma, the more complex the model, the higher the risk of overfitting.
  
  In fact, as you can read on the page you linked:
  
  > If **gamma is too large**, the radius of the area of influence of the
  > support vectors **only includes the support vector itself**[...]
  >
  > When **gamma is very small**, the model is too constrained and **cannot
  > capture the complexity or “shape” of the data**. The **region of influence
  > of any selected support vector would include the whole training set**.
  
  > **Comments:**
  > **username_0**: i not understanding this "of a single training example reaches"  
  > **username_3**: Replace "a single" with "each individual".  
  
  ---
  
  