<p>The convolutions are linear transformations. However in typical applications a non linear activation function like RELU is used following the convolution to provide non-linearity otherwise a convolutional neural network would just be a net linear transformation.</p>

CC BY-SA 4.0
2020-03-31T06:33:24.717Z
2020-03-31T06:33:24.717Z
Gerry P
<p><a href="https://i.sstatic.net/7iBKO.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/7iBKO.png" alt="enter image description here"></a></p>

<p>The image above, a screenshot from <a href="https://towardsdatascience.com/intuitively-understanding-convolutions-for-deep-learning-1f6f42faee1" rel="nofollow noreferrer">this article</a>, describes discrete 2D convolutions as <em>linear transforms.</em> The idea used, as far as I understand, is to represent the 2 dimensional <span class="math-container">$n$</span>x<span class="math-container">$n$</span> input grid as a vector of <span class="math-container">$n^2$</span> length, and the <span class="math-container">$m$</span>x<span class="math-container">$m$</span> output grid as a vector of <span class="math-container">$m^2$</span> length. I don't see why this can't be generalised to higher-dimensional convolutions, since a transformation matrix can be constructed from one vector to another, no matter the length (<strong>right?</strong>)</p>

<p>My question: <strong>Aren't all discrete convolutions (not just 2D) linear transforms?</strong></p>

<p>Are there cases where such a transformation matrix cannot be found?</p>

CC BY-SA 4.0
2020-03-30T18:16:16.093Z
What do you mean by "higher-dimensional convolutions"? Can you provide a link to a definition of them?
nbro
CC BY-SA 4.0
2020-03-31T03:46:47.750Z
Could you elaborate, @DuttaA ? Maybe post it as an answer.
stoic-santiago
CC BY-SA 4.0
2020-03-31T03:48:29.357Z
@nbro I would think of higher dimensional convolutional as starting with a 3D grid of input, say $n$x$n$x$n$, and mapping it to a smaller 3D grid using a kernel - same as in the case of 2D grids. This may generalise to higher dimensions, even.
stoic-santiago
<convolutional-neural-networks><signal-processing>
Aren't all discrete convolutions (not just 2D) linear transforms?
stoic-santiago
# Aren't all discrete convolutions (not just 2D) linear transforms?

(Asked by: username_0 on 2020-03-30)

[![enter image description here](https://i.sstatic.net/7iBKO.png)](https://i.sstatic.net/7iBKO.png)

The image above, a screenshot from [this article](https://towardsdatascience.com/intuitively-understanding-convolutions-for-deep-learning-1f6f42faee1), describes discrete 2D convolutions as *linear transforms.* The idea used, as far as I understand, is to represent the 2 dimensional $n$x$n$ input grid as a vector of $n^2$ length, and the $m$x$m$ output grid as a vector of $m^2$ length. I don't see why this can't be generalised to higher-dimensional convolutions, since a transformation matrix can be constructed from one vector to another, no matter the length (**right?**)

My question: **Aren't all discrete convolutions (not just 2D) linear transforms?**

Are there cases where such a transformation matrix cannot be found?

**Tags:** convolutional-neural-networks, signal-processing

> **Comments:**
> **username_1**: What do you mean by "higher-dimensional convolutions"? Can you provide a link to a definition of them?  
> **username_0**: Could you elaborate, @DuttaA ? Maybe post it as an answer.  
> **username_0**: @username_1 I would think of higher dimensional convolutional as starting with a 3D grid of input, say $n$x$n$x$n$, and mapping it to a smaller 3D grid using a kernel - same as in the case of 2D grids. This may generalise to higher dimensions, even.  

---

## 1 answers

### Answer by username_2 on 2020-03-31. Score: 1

The convolutions are linear transformations. However in typical applications a non linear activation function like RELU is used following the convolution to provide non-linearity otherwise a convolutional neural network would just be a net linear transformation.


---

