Answers.Body: <p>The convolutions are linear transformations. However in typical applications a non linear activation function like RELU is used following the convolution to provide non-linearity otherwise a convolutional neural network would just be a net linear transformation.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-31T06:33:24.717Z
Answers.LastActivityDate: 2020-03-31T06:33:24.717Z
Answers.UserDisplayName: Gerry P
Body: <p><a href="https://i.sstatic.net/7iBKO.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/7iBKO.png" alt="enter image description here"></a></p>
  
  <p>The image above, a screenshot from <a href="https://towardsdatascience.com/intuitively-understanding-convolutions-for-deep-learning-1f6f42faee1" rel="nofollow noreferrer">this article</a>, describes discrete 2D convolutions as <em>linear transforms.</em> The idea used, as far as I understand, is to represent the 2 dimensional <span class="math-container">$n$</span>x<span class="math-container">$n$</span> input grid as a vector of <span class="math-container">$n^2$</span> length, and the <span class="math-container">$m$</span>x<span class="math-container">$m$</span> output grid as a vector of <span class="math-container">$m^2$</span> length. I don't see why this can't be generalised to higher-dimensional convolutions, since a transformation matrix can be constructed from one vector to another, no matter the length (<strong>right?</strong>)</p>
  
  <p>My question: <strong>Aren't all discrete convolutions (not just 2D) linear transforms?</strong></p>
  
  <p>Are there cases where such a transformation matrix cannot be found?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-30T18:16:16.093Z
Comments.Text: What do you mean by "higher-dimensional convolutions"? Can you provide a link to a definition of them?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-31T03:46:47.750Z
Comments.Text: Could you elaborate, @DuttaA ? Maybe post it as an answer.
Comments.UserDisplayName: stoic-santiago
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-31T03:48:29.357Z
Comments.Text: @nbro I would think of higher dimensional convolutional as starting with a 3D grid of input, say $n$x$n$x$n$, and mapping it to a smaller 3D grid using a kernel - same as in the case of 2D grids. This may generalise to higher dimensions, even.
Comments.UserDisplayName: stoic-santiago
Tags: <convolutional-neural-networks><signal-processing>
Title: Aren't all discrete convolutions (not just 2D) linear transforms?
UserDisplayName: stoic-santiago
ThreadText: # Aren't all discrete convolutions (not just 2D) linear transforms?
  
  (Asked by: username_0 on 2020-03-30)
  
  [![enter image description here](https://i.sstatic.net/7iBKO.png)](https://i.sstatic.net/7iBKO.png)
  
  The image above, a screenshot from [this article](https://towardsdatascience.com/intuitively-understanding-convolutions-for-deep-learning-1f6f42faee1), describes discrete 2D convolutions as *linear transforms.* The idea used, as far as I understand, is to represent the 2 dimensional $n$x$n$ input grid as a vector of $n^2$ length, and the $m$x$m$ output grid as a vector of $m^2$ length. I don't see why this can't be generalised to higher-dimensional convolutions, since a transformation matrix can be constructed from one vector to another, no matter the length (**right?**)
  
  My question: **Aren't all discrete convolutions (not just 2D) linear transforms?**
  
  Are there cases where such a transformation matrix cannot be found?
  
  **Tags:** convolutional-neural-networks, signal-processing
  
  > **Comments:**
  > **username_1**: What do you mean by "higher-dimensional convolutions"? Can you provide a link to a definition of them?  
  > **username_0**: Could you elaborate, @DuttaA ? Maybe post it as an answer.  
  > **username_0**: @username_1 I would think of higher dimensional convolutional as starting with a 3D grid of input, say $n$x$n$x$n$, and mapping it to a smaller 3D grid using a kernel - same as in the case of 2D grids. This may generalise to higher dimensions, even.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-03-31. Score: 1
  
  The convolutions are linear transformations. However in typical applications a non linear activation function like RELU is used following the convolution to provide non-linearity otherwise a convolutional neural network would just be a net linear transformation.
  
  
  ---
  
  