Answers.Body: <p>Mathematically, the convolution is an operation that takes two functions, <span class="math-container">$f$</span> and <span class="math-container">$g$</span>, and produces a third function, <span class="math-container">$h$</span>. Concisely, we can denote the convolution operation as follows</p>
  <p><span class="math-container">$$f \circledast g = h$$</span></p>
  <p>In the context of computer vision and, in particular, image processing, the convolution is widely used to apply a so-called <em>kernel</em> (aka filter) to an <em>input</em> (typically, an image, but this does not have to be the case). The input (e.g. an image), the kernel, and the output of the convolution, in this context, is usually a matrix or a tensor. In image processing, the convolution is typically used to e.g. blur images or maybe to remove noise.</p>
  <p>However, in the beginning, I said that the convolution is an operation that takes two functions (and not matrices) and produces a third one, so these two explanations of the convolution do not seem to be consistent, right?</p>
  <p>The answer to this question is that the two explanations are consistent with each other. More precisely, if you have a function <span class="math-container">$f : X \rightarrow Y$</span> (assuming that <span class="math-container">$X$</span> is discrete/countable), you can represent it in a vector form as follows <span class="math-container">$\mathbf{f} = [y_1, y_2, \dots, y_n]$</span>, i.e. <span class="math-container">$\mathbf{f}$</span> is a vector that contains all outputs of the function <span class="math-container">$f$</span> (for all possible inputs).</p>
  <p>In image processing, an image and a kernel can also be thought of as a function with a discrete domain (i.e. the pixels), so the matrices that represent the image or the kernel are just the vector forms of the corresponding functions. See <a href="https://ai.stackexchange.com/a/21401/2444">this answer</a> for more details about representing an image as a function.</p>
  <p>Once you understand that the convolution in image processing is really the convolution operation as defined in mathematics, then you can simply look up the mathematical definition of the convolution operation.</p>
  <p>In the discrete case (i.e. you can think of the function as vectors, as explained above), the convolution is defined as</p>
  <p><span class="math-container">$${\displaystyle h[n] =  (f  \circledast g)[n]=\sum _{m=-M}^{M}f[n-m]g[m].} \tag{1}\label{1}$$</span></p>
  <p>You can read equation <span class="math-container">$1$</span> as follows</p>
  <ul>
  <li><span class="math-container">$f  \circledast g$</span> is the convolution of the input function (or matrix) <span class="math-container">$f$</span> and the kernel <span class="math-container">$g$</span></li>
  <li><span class="math-container">$(f  \circledast g)[n]$</span> is the output of the convolution <span class="math-container">$f  \circledast g$</span> at index (or input position) <span class="math-container">$n$</span> (so you need to apply equation \ref{1} for all <span class="math-container">$n$</span>, if you want to have <span class="math-container">$h$</span> and not just <span class="math-container">$h[n]$</span>)</li>
  <li>So, the result of the convolution at <span class="math-container">$n$</span>, <span class="math-container">$h[n]$</span>, is defined as <span class="math-container">$\sum _{m=-M}^{M}f[n-m]g[m]$</span>, a sum that goes from <span class="math-container">$m = -M$</span> to <span class="math-container">$m = M$</span>. Here <span class="math-container">$M$</span> may be half of the length of the kernel matrix. For example, if you use the following Gaussian kernel, then <span class="math-container">$M = 2$</span> (and I assume that the center of the kernel is at coordinate <span class="math-container">$(0, 0)$</span>).</li>
  </ul>
  <p><span class="math-container">$$
  \mathbf{g} = 
  \frac{1}{273}
  \begin{bmatrix}
  1 &amp; 4 &amp; 7 &amp; 4 &amp; 1 \\
  4 &amp; 16 &amp; 26 &amp; 16 &amp; 4 \\
  7 &amp; 26 &amp; 41 &amp; 26 &amp; 7 \\
  4 &amp; 16 &amp; 26 &amp; 16 &amp; 4 \\
  1 &amp; 4 &amp; 7 &amp; 4 &amp; 1
  \end{bmatrix}
  \label{2}\tag{2}
  $$</span></p>
  <p>Here are some notes:</p>
  <ul>
  <li><p>The kernel \ref{2} is symmetric around the <span class="math-container">$x$</span> and <span class="math-container">$y$</span> axes: this actually implies that the convolution is equal to the cross-correlation, so you don't even have to worry about their equivalence or not (in case you have ever worried about it, which would have happened only if you already came across the cross-correlation). See <a href="https://ai.stackexchange.com/q/21999/2444">this question</a> for more info.</p>
  </li>
  <li><p>The kernel \ref{2} is the vector form of the function form of the 2d Gaussian kernel (the one in your question): more precisely, an integer-valued approximation of the 2D Gaussian kernel when <span class="math-container">$\sigma = 1$</span> (as stated in your slides).</p>
  </li>
  <li><p>The convolution can be implemented as matrix multiplication. This may not be useful now, but it's something useful to know if you want to implement it. See <a href="https://ai.stackexchange.com/q/11172/2444">this question</a> for more info.</p>
  </li>
  </ul>
  <p>Question for you: what is the result of the application of this Gaussian kernel to any input? What does this kernel intuitively do? Once you fully understand the convolution, you can answer this question.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-09-30T06:04:20.753Z
Answers.Text: Thank you for clear answer, this is exactly what I needed to further understand the convolution! So the kernel intuitively multiplies corresponding elements of the input vector with kernel values and count it together, since my matrix is very sparse in most cases and radius arround non-zero element are often only 0s, the result is only multiplication of the one (or few) non-zero element plus 0 times rest.
Answers.UserDisplayName: Ruli
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-09-30T13:08:59.243Z
Answers.Text: @Ruli Note that if you use a matrix instead of a vector (to represent the input and kernel), you will need 2 sums (one that goes horizontally across the kernel and image and one that goes vertically) in the definition of the discrete convolution (rather than just 1, like I wrote above, which is the definition for 1-dimensional signals, i.e. vectors). Maybe [this Wikipedia article](https://en.wikipedia.org/wiki/Multidimensional_discrete_convolution) will clarify more what I mean. Maybe I will edit my answer later to be more precise and include this info.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-09-29T17:35:39.490Z
Answers.LastActivityDate: 2020-09-29T17:54:55.420Z
Answers.LastEditDate: 2020-09-29T17:54:55.420Z
Answers.UserDisplayName: nbro
Body: <p>I have to build a model where I pre-process the data with a Gaussian kernel. The data are an <span class="math-container">$n\times n$</span> matrix (i.e one channel), but not an image, thus I can't refer to this matrix as an image and to its elements as pixels. The Gaussian kernel is built by the following function (more i.e. <a href="https://www.cs.auckland.ac.nz/courses/compsci373s1c/PatricesLectures/Gaussian%20Filtering_1up.pdf" rel="nofollow noreferrer">here</a>)</p>
  <p><span class="math-container">$$\begin{equation}
  \begin{aligned}
  g(x,y,\sigma) = 
  \dfrac{1}{2\pi\sigma^2} e^{\dfrac{-(x^2+y^2)}{2\sigma^2}}.
  \end{aligned}
  \end{equation}$$</span></p>
  <p>This kernel is moving one by one element and doing convolution. In my case, most of the elements are zero, the matrix is sparse.</p>
  <p>How can I describe/understand the process of convolving the original data with a Gaussian kernel?</p>
  <p>I have been looking for some articles, but I am unable to find any mathematical explanations, only explanation in words or pseudo-code.</p>
  
Tags: <convolutional-neural-networks><math><image-processing><data-preprocessing><convolution>
Title: How to mathematically describe the convolution operation (with a Gaussian kernel)?
UserDisplayName: Ruli
ThreadText: # How to mathematically describe the convolution operation (with a Gaussian kernel)?
  
  (Asked by: username_0 on 2020-09-29)
  
  I have to build a model where I pre-process the data with a Gaussian kernel. The data are an $n\times n$ matrix (i.e one channel), but not an image, thus I can't refer to this matrix as an image and to its elements as pixels. The Gaussian kernel is built by the following function (more i.e. [here](https://www.cs.auckland.ac.nz/courses/compsci373s1c/PatricesLectures/Gaussian%20Filtering_1up.pdf))
  
  $$\begin{equation}
  \begin{aligned}
  g(x,y,\sigma) =
  \dfrac{1}{2\pi\sigma^2} e^{\dfrac{-(x^2+y^2)}{2\sigma^2}}.
  \end{aligned}
  \end{equation}$$
  
  This kernel is moving one by one element and doing convolution. In my case, most of the elements are zero, the matrix is sparse.
  
  How can I describe/understand the process of convolving the original data with a Gaussian kernel?
  
  I have been looking for some articles, but I am unable to find any mathematical explanations, only explanation in words or pseudo-code.
  
  **Tags:** convolutional-neural-networks, math, image-processing, data-preprocessing, convolution
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-09-29. Score: 2 | Accepted answer
  
  Mathematically, the convolution is an operation that takes two functions, $f$ and $g$, and produces a third function, $h$. Concisely, we can denote the convolution operation as follows
  
  $$f \circledast g = h$$
  
  In the context of computer vision and, in particular, image processing, the convolution is widely used to apply a so-called *kernel* (aka filter) to an *input* (typically, an image, but this does not have to be the case). The input (e.g. an image), the kernel, and the output of the convolution, in this context, is usually a matrix or a tensor. In image processing, the convolution is typically used to e.g. blur images or maybe to remove noise.
  
  However, in the beginning, I said that the convolution is an operation that takes two functions (and not matrices) and produces a third one, so these two explanations of the convolution do not seem to be consistent, right?
  
  The answer to this question is that the two explanations are consistent with each other. More precisely, if you have a function $f : X \rightarrow Y$ (assuming that $X$ is discrete/countable), you can represent it in a vector form as follows $\mathbf{f} = [y\_1, y\_2, \dots, y\_n]$, i.e. $\mathbf{f}$ is a vector that contains all outputs of the function $f$ (for all possible inputs).
  
  In image processing, an image and a kernel can also be thought of as a function with a discrete domain (i.e. the pixels), so the matrices that represent the image or the kernel are just the vector forms of the corresponding functions. See [this answer](https://ai.stackexchange.com/a/21401/2444) for more details about representing an image as a function.
  
  Once you understand that the convolution in image processing is really the convolution operation as defined in mathematics, then you can simply look up the mathematical definition of the convolution operation.
  
  In the discrete case (i.e. you can think of the function as vectors, as explained above), the convolution is defined as
  
  $${\displaystyle h[n] = (f \circledast g)[n]=\sum \_{m=-M}^{M}f[n-m]g[m].} \tag{1}\label{1}$$
  
  You can read equation $1$ as follows
  
  * $f \circledast g$ is the convolution of the input function (or matrix) $f$ and the kernel $g$
  * $(f \circledast g)[n]$ is the output of the convolution $f \circledast g$ at index (or input position) $n$ (so you need to apply equation \ref{1} for all $n$, if you want to have $h$ and not just $h[n]$)
  * So, the result of the convolution at $n$, $h[n]$, is defined as $\sum \_{m=-M}^{M}f[n-m]g[m]$, a sum that goes from $m = -M$ to $m = M$. Here $M$ may be half of the length of the kernel matrix. For example, if you use the following Gaussian kernel, then $M = 2$ (and I assume that the center of the kernel is at coordinate $(0, 0)$).
  
  $$
  \mathbf{g} =
  \frac{1}{273}
  \begin{bmatrix}
  1 & 4 & 7 & 4 & 1 \\
  4 & 16 & 26 & 16 & 4 \\
  7 & 26 & 41 & 26 & 7 \\
  4 & 16 & 26 & 16 & 4 \\
  1 & 4 & 7 & 4 & 1
  \end{bmatrix}
  \label{2}\tag{2}
  $$
  
  Here are some notes:
  
  * The kernel \ref{2} is symmetric around the $x$ and $y$ axes: this actually implies that the convolution is equal to the cross-correlation, so you don't even have to worry about their equivalence or not (in case you have ever worried about it, which would have happened only if you already came across the cross-correlation). See [this question](https://ai.stackexchange.com/q/21999/2444) for more info.
  * The kernel \ref{2} is the vector form of the function form of the 2d Gaussian kernel (the one in your question): more precisely, an integer-valued approximation of the 2D Gaussian kernel when $\sigma = 1$ (as stated in your slides).
  * The convolution can be implemented as matrix multiplication. This may not be useful now, but it's something useful to know if you want to implement it. See [this question](https://ai.stackexchange.com/q/11172/2444) for more info.
  
  Question for you: what is the result of the application of this Gaussian kernel to any input? What does this kernel intuitively do? Once you fully understand the convolution, you can answer this question.
  
  > **Comments:**
  > **username_0**: Thank you for clear answer, this is exactly what I needed to further understand the convolution! So the kernel intuitively multiplies corresponding elements of the input vector with kernel values and count it together, since my matrix is very sparse in most cases and radius arround non-zero element are often only 0s, the result is only multiplication of the one (or few) non-zero element plus 0 times rest.  
  > **username_1**: @Ruli Note that if you use a matrix instead of a vector (to represent the input and kernel), you will need 2 sums (one that goes horizontally across the kernel and image and one that goes vertically) in the definition of the discrete convolution (rather than just 1, like I wrote above, which is the definition for 1-dimensional signals, i.e. vectors). Maybe [this Wikipedia article](https://en.wikipedia.org/wiki/Multidimensional_discrete_convolution) will clarify more what I mean. Maybe I will edit my answer later to be more precise and include this info.  
  
  ---
  
  