Answers.Body: <p><span class="math-container">$\mathbf{x} \in \mathbb{R}^p$</span> and <span class="math-container">$\mathbf{x}' \in \mathbb{R}^p$</span> are two inputs (or feature vectors). </p>
  
  <p>In the context of classification with an SVM, you are given a dataset <span class="math-container">$D = \{(\mathbf{x}_i, y_i) \}_{i=1}^N$</span>, where <span class="math-container">$\mathbf{x}_i \in \mathbb{R}^p$</span> is an input (or point) and <span class="math-container">$y_i$</span> the corresponding label. The goal is to find a hyperplane that classifies the points <span class="math-container">$\mathbf{x}_i$</span>. The hyperplane actually corresponds to a binary classifier that splits the plane into two, so the assumption is that there are two labels. However, these points <span class="math-container">$ \mathbf{x}_i$</span> may not be linearly separable in <span class="math-container">$\mathbb{R}^p$</span>, i.e. there may not be a hyperplane (in 2d, i.e. when <span class="math-container">$p=2$</span>, a hyperplane is a line) that separates them. The kernel trick, i.e. the use of kernels (such as the Gaussian radial basis), allows an SVM to perform non-linear classification by transforming the inputs to a space where they are linearly separable.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-01T15:47:20.933Z
Answers.Text: Just to solidify my understanding, 
  if I take the MNIST dataset for example and reduce it to just working with 2 classes (instead of the actual 10), with images of size 28x28 and say there are 500 examples in this example dataset. I would then have a 500x28x28 dataset with corresponding y labels of 500x2.
  If I then wanted to apply this gaussian kernel to this dataset, what are the actual values I would plug in?
Answers.UserDisplayName: FeedMeInformation
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-01T15:59:36.653Z
Answers.Text: In your example, $\mathbf{x}$ (or  $\mathbf{x}'$) would be a MNIST image, which can be represented as a vector in $\mathbb{R}^{784}$ (where $784 = 28*28$). In your example, you would have $500$ examples of the form $\mathbf{x}$. The labels don't have to be a vector, they can simply be a scalar, so your labels could simply be a vector of the form $\mathbb{R}^{500}$. Regarding how you apply the kernel, this is more a detail of how the SVM works, but, to give you the intuition, essentially, you will be replacing the dot products in the SVM computations with $k(\mathbf{x}, \mathbf{x}')$.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-01T18:08:14.330Z
Answers.Text: so is x and x' two of the datapoints(image vectors) in my dataset? sorry but its just not clicking.
Answers.UserDisplayName: FeedMeInformation
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-01T18:09:51.100Z
Answers.Text: @FeedMeInformation Yes. (But the application of a kernel function $k$ is not just restricted to the data points (image vectors), but, given that it is a general function, it can be applied to any pair of vectors. You can ignore this, if you don't want to know the details of SVMs. But, if you read the Wikipedia entry on SVMs, you will get a better feeling of the whole picture (even though Wikipedia may not be the most reliable source ever)).
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-01T18:13:57.667Z
Answers.Text: @FeedMeInformation In the SVM formulations, you will see dot products, which, in general, are often used to calculate a notion of "distance" between vectors. The kernel function (e.g. your Gaussian radial basis) has the same purpose (in this and other contexts), i.e. it's still a way of calculating distances between vectors, but you do it in a more "sophisticated way". So, you can think of a kernel function as a function that computes a notion of "distance".
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-01T13:21:29.247Z
Answers.LastActivityDate: 2020-03-01T13:26:45.127Z
Answers.LastEditDate: 2020-03-01T13:26:45.127Z
Answers.UserDisplayName: nbro
Body: <p>If I have the Gaussian kernel</p>
  
  <p><span class="math-container">$$
  k(x, x') = \operatorname{exp}\left( -\| x - x' \|^2 / 2\sigma^2 \right)
  $$</span></p>
  
  <p>What is <span class="math-container">$x$</span> and <span class="math-container">$x'$</span> in the context of training an SVM?</p>
  
Tags: <machine-learning><math><support-vector-machine>
Title: What are the variables used in a Gaussian radial basis kernel in the context of SVMs?
UserDisplayName: FeedMeInformation
ThreadText: # What are the variables used in a Gaussian radial basis kernel in the context of SVMs?
  
  (Asked by: username_0 on 2020-03-01)
  
  If I have the Gaussian kernel
  
  $$
  k(x, x') = \operatorname{exp}\left( -\| x - x' \|^2 / 2\sigma^2 \right)
  $$
  
  What is $x$ and $x'$ in the context of training an SVM?
  
  **Tags:** machine-learning, math, support-vector-machine
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-03-01. Score: 1 | Accepted answer
  
  $\mathbf{x} \in \mathbb{R}^p$ and $\mathbf{x}' \in \mathbb{R}^p$ are two inputs (or feature vectors).
  
  In the context of classification with an SVM, you are given a dataset $D = \{(\mathbf{x}\_i, y\_i) \}\_{i=1}^N$, where $\mathbf{x}\_i \in \mathbb{R}^p$ is an input (or point) and $y\_i$ the corresponding label. The goal is to find a hyperplane that classifies the points $\mathbf{x}\_i$. The hyperplane actually corresponds to a binary classifier that splits the plane into two, so the assumption is that there are two labels. However, these points $ \mathbf{x}\_i$ may not be linearly separable in $\mathbb{R}^p$, i.e. there may not be a hyperplane (in 2d, i.e. when $p=2$, a hyperplane is a line) that separates them. The kernel trick, i.e. the use of kernels (such as the Gaussian radial basis), allows an SVM to perform non-linear classification by transforming the inputs to a space where they are linearly separable.
  
  > **Comments:**
  > **username_0**: Just to solidify my understanding,  if I take the MNIST dataset for example and reduce it to just working with 2 classes (instead of the actual 10), with images of size 28x28 and say there are 500 examples in this example dataset. I would then have a 500x28x28 dataset with corresponding y labels of 500x2. If I then wanted to apply this gaussian kernel to this dataset, what are the actual values I would plug in?  
  > **username_1**: In your example, $\mathbf{x}$ (or  $\mathbf{x}'$) would be a MNIST image, which can be represented as a vector in $\mathbb{R}^{784}$ (where $784 = 28*28$). In your example, you would have $500$ examples of the form $\mathbf{x}$. The labels don't have to be a vector, they can simply be a scalar, so your labels could simply be a vector of the form $\mathbb{R}^{500}$. Regarding how you apply the kernel, this is more a detail of how the SVM works, but, to give you the intuition, essentially, you will be replacing the dot products in the SVM computations with $k(\mathbf{x}, \mathbf{x}')$.  
  > **username_0**: so is x and x' two of the datapoints(image vectors) in my dataset? sorry but its just not clicking.  
  > **username_1**: @FeedMeInformation Yes. (But the application of a kernel function $k$ is not just restricted to the data points (image vectors), but, given that it is a general function, it can be applied to any pair of vectors. You can ignore this, if you don't want to know the details of SVMs. But, if you read the Wikipedia entry on SVMs, you will get a better feeling of the whole picture (even though Wikipedia may not be the most reliable source ever)).  
  > **username_1**: @FeedMeInformation In the SVM formulations, you will see dot products, which, in general, are often used to calculate a notion of "distance" between vectors. The kernel function (e.g. your Gaussian radial basis) has the same purpose (in this and other contexts), i.e. it's still a way of calculating distances between vectors, but you do it in a more "sophisticated way". So, you can think of a kernel function as a function that computes a notion of "distance".  
  
  ---
  
  