Answers.Body: <p>The dimensionality used to discuss convolutional layers in CNNs is based on the dimensionality of the input <em>without considering channels</em>.</p>
  <ul>
  <li>1D CNNs might process raw audio sources (mono or stereo), text sequences, IR spectrometry from a single sample point</li>
  <li>2D CNNs can process photographic images (regardless of colour/depth etc information), audio spectrograms, grid-based board games</li>
  <li>3D CNNs can process voxels from Minecraft, image sequences from videos etc</li>
  </ul>
  <p>It is often possible to perform signal processing that changes dimensions of signal sources. Whether that adds &quot;channels&quot; or adds a dimension can be a matter of convenience to fit a particular approach. In terms of defining a n-dimensional array, then the addition of channels is just another dimension. In terms of considering signal processing performed in CNNs, we care about the distinction between channels and the rest of the space that the signal exists in.</p>
  <p>One way to decide whether something is considered a channel or a CNN layer dimension is whether there is an ordering or metric that consistently separates measurements over that dimension. If a metric such as space, time or frequency applies, then that dimension can be considered part of the &quot;core&quot; dimensionality that defines the problem, whilst a more arbitrary set of features (e.g. each entry in the vector embedding of a word) is more channel-like.</p>
  <p>As standard CNN design involves summing over all input channels to create each output feature/channel, which is mathematically the same as increasing the convolution dimension (when the kernel size in that dimension matches to the number of channels), then in practice the convolution <em>operation</em> implemented in a CNN layer of a particular dimensionality can be one dimension size higher. E.g. a layer class labelled &quot;Conv1D&quot; will perform a 2D convolution operation, with the added dimension size matching exactly to the number of input channels. However, conceptually it makes sense to view this as a sum of lower-dimension convolutions, because of the need to exactly match the dimension size. This extra dimension is seen as a convenience for calculation, and not part of the definition.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-08T13:51:36.607Z
Answers.LastActivityDate: 2021-10-08T14:23:56.300Z
Answers.LastEditDate: 2021-10-08T14:23:56.300Z
Answers.UserDisplayName: Neil Slater
Answers.Body: <p>Kernel dimensionality and presence of filters decides the dimension of convolution operator. N Dimensional Convolutions have N dimensional kernels. For example, from Keras Documentation on <a href="https://keras.io/api/layers/convolution_layers/convolution2d/" rel="nofollow noreferrer">2 Dimensional Convolutions</a>:</p>
  <blockquote>
  <p>kernel_size: An integer or tuple/list of 2 integers, specifying the height and width of the 2D convolution window. Can be a single integer to specify the same value for all spatial dimensions.</p>
  </blockquote>
  <p>If you have more than one filter in the layer, that also adds another dimension to the layer. So, we can say that a 2D convolutional layer is in general 3 dimensional, where 3rd dimension is the number of filters: (k,k,F). For the special case of a single filter, F=1 and we can treat it as 2 dimensional.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-01T11:35:06.327Z
Answers.Text: Please check [this](https://ai.stackexchange.com/a/29977/18758) answer
Answers.UserDisplayName: hanugm
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-01T11:58:49.853Z
Answers.Text: @hanugm Yes, that is the case in the first layer for a color image, a separate kernel used for each color. Still, each kernel is 2 dimensional. And if we have an additional convolutional layer with 16 filter, we still have 2 dimensional kernels, but now 16 of them.
Answers.UserDisplayName: serali
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-08T13:40:15.193Z
Answers.Text: Worth noting that for efficiency purposes a Conv2D layer is very likely running a 3D convolution operation when feeding forward. This is used to sum over the input channels. However, it is still useful to think of this as conceptually a sum of 2D convolutions when looking from the perspective of signal processing from the input.
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-01T11:00:35.733Z
Answers.LastActivityDate: 2021-10-01T13:03:39.577Z
Answers.LastEditDate: 2021-10-01T13:03:39.577Z
Answers.UserDisplayName: serali
Body: <p>In practical applications, we generally talk about three types of convolution layers: 1-dimensional convolution, 2-dimensional convolution, and 3-dimensional convolution. Most popular packages like PyTorch, Keras, etc., provide Conv1d, Conv2d, and Conv3d.</p>
  <p>What is the deciding factor for the dimensionality of the convolution layer mentioned in the packages?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-10-01T15:23:11.753Z
Comments.Text: What do you mean by "dimensionality of the convolution layer"? A convolutional layer is just **an abstraction to perform the convolution** (i.e. a function). So, to me, it doesn't make sense to ask "What is the dimensionality of a function?". So, you should start by clarifying this before proceeding. Also, I don't understand this sentence "Neither the shape of the input nor the shape of the kernel decides the dimensionality of the convolution operation." because I don't understand what you mean by "dimensionality of a layer". So, you should also clarify that sentence and why you claim that.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-10-07T10:09:59.510Z
Comments.Text: @nbro Is it fine now? I am asking about the dimensionality of the layers that are discussed in packages.
Comments.UserDisplayName: hanugm
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-10-07T14:51:59.470Z
Comments.Text: To be honest, your last edit doesn't clarify what was in my opinion unclear. As I said, convolutional layers should be seen more as functions rather than tensors (indeed, they are just an implementation of a function, i.e. the convolution). So, I don't understand what you mean by "dimensionality of a function here". Maybe you mean "the dimensionality of the input to and output of the convolutional layer" That would make more sense. Like in the function $f : \mathbb{R}^n \rightarrow \mathbb{R}$. If that's not what you mean, I don't understand what you mean.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-10-07T14:55:25.177Z
Comments.Text: If you're asking about whether we use 3d or 2d convolutions, then that's a different story than what your post seems to suggest. If we use 3d or 2d convolutions, it's a matter of choice of the programmer (usually). Anyway, please, edit your post to clarify what you really mean by "dimensionality of the convolutional layer".
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-10-08T13:17:07.340Z
Comments.Text: Are you now asking "What do people refer to when they use the word 'dimensionality' in the context of convolutional layer"? If yes, I would suggest that you remove the sentences "The dimensionality I am referring to is obvious from this context." and "Neither the shape of the input nor the shape of the kernel decides the dimensionality of the convolution operation.". It may also be a good idea to quote someone that used this terminology, i.e. "dimensionality of convolutional layer". The context will probably be very useful to answer that question.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-10-08T20:05:28.920Z
Comments.Text: Ok, now, the question is clearer to me, but still not completely clear. Are you asking 1. how we choose between conv1d, conv2d, and conv3d, or 2. what makes, for example, a conv2d and 2d convolution rather than a 1d or 3d convolutional, i.e. why is e.g. a conv2d a convolution in 2d? These questions are related, but different, I guess.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-10-08T21:42:04.280Z
Comments.Text: I am asking for 2. Which element/entity decides 1d,2d or 3d. @nbro
Comments.UserDisplayName: hanugm
Tags: <convolutional-neural-networks><convolutional-layers><1d-convolution><2d-convolution><3d-convolution>
Title: What do people refer to when they use the word 'dimensionality' in the context of convolutional layer?
UserDisplayName: hanugm
ThreadText: # What do people refer to when they use the word 'dimensionality' in the context of convolutional layer?
  
  (Asked by: username_0 on 2021-10-01)
  
  In practical applications, we generally talk about three types of convolution layers: 1-dimensional convolution, 2-dimensional convolution, and 3-dimensional convolution. Most popular packages like PyTorch, Keras, etc., provide Conv1d, Conv2d, and Conv3d.
  
  What is the deciding factor for the dimensionality of the convolution layer mentioned in the packages?
  
  **Tags:** convolutional-neural-networks, convolutional-layers, 1d-convolution, 2d-convolution, 3d-convolution
  > **Comments:**
  > **username_1**: What do you mean by "dimensionality of the convolution layer"? A convolutional layer is just **an abstraction to perform the convolution** (i.e. a function). So, to me, it doesn't make sense to ask "What is the dimensionality of a function?". So, you should start by clarifying this before proceeding. Also, I don't understand this sentence "Neither the shape of the input nor the shape of the kernel decides the dimensionality of the convolution operation." because I don't understand what you mean by "dimensionality of a layer". So, you should also clarify that sentence and why you claim that.  
  > **username_0**: @nbro Is it fine now? I am asking about the dimensionality of the layers that are discussed in packages.  
  > **username_1**: To be honest, your last edit doesn't clarify what was in my opinion unclear. As I said, convolutional layers should be seen more as functions rather than tensors (indeed, they are just an implementation of a function, i.e. the convolution). So, I don't understand what you mean by "dimensionality of a function here". Maybe you mean "the dimensionality of the input to and output of the convolutional layer" That would make more sense. Like in the function $f : \mathbb{R}^n \rightarrow \mathbb{R}$. If that's not what you mean, I don't understand what you mean.  
  > **username_1**: If you're asking about whether we use 3d or 2d convolutions, then that's a different story than what your post seems to suggest. If we use 3d or 2d convolutions, it's a matter of choice of the programmer (usually). Anyway, please, edit your post to clarify what you really mean by "dimensionality of the convolutional layer".  
  > **username_1**: Are you now asking "What do people refer to when they use the word 'dimensionality' in the context of convolutional layer"? If yes, I would suggest that you remove the sentences "The dimensionality I am referring to is obvious from this context." and "Neither the shape of the input nor the shape of the kernel decides the dimensionality of the convolution operation.". It may also be a good idea to quote someone that used this terminology, i.e. "dimensionality of convolutional layer". The context will probably be very useful to answer that question.  
  > **username_1**: Ok, now, the question is clearer to me, but still not completely clear. Are you asking 1. how we choose between conv1d, conv2d, and conv3d, or 2. what makes, for example, a conv2d and 2d convolution rather than a 1d or 3d convolutional, i.e. why is e.g. a conv2d a convolution in 2d? These questions are related, but different, I guess.  
  > **username_0**: I am asking for 2. Which element/entity decides 1d,2d or 3d. @nbro  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2021-10-08. Score: 1
  
  The dimensionality used to discuss convolutional layers in CNNs is based on the dimensionality of the input *without considering channels*.
  
  * 1D CNNs might process raw audio sources (mono or stereo), text sequences, IR spectrometry from a single sample point
  * 2D CNNs can process photographic images (regardless of colour/depth etc information), audio spectrograms, grid-based board games
  * 3D CNNs can process voxels from Minecraft, image sequences from videos etc
  
  It is often possible to perform signal processing that changes dimensions of signal sources. Whether that adds "channels" or adds a dimension can be a matter of convenience to fit a particular approach. In terms of defining a n-dimensional array, then the addition of channels is just another dimension. In terms of considering signal processing performed in CNNs, we care about the distinction between channels and the rest of the space that the signal exists in.
  
  One way to decide whether something is considered a channel or a CNN layer dimension is whether there is an ordering or metric that consistently separates measurements over that dimension. If a metric such as space, time or frequency applies, then that dimension can be considered part of the "core" dimensionality that defines the problem, whilst a more arbitrary set of features (e.g. each entry in the vector embedding of a word) is more channel-like.
  
  As standard CNN design involves summing over all input channels to create each output feature/channel, which is mathematically the same as increasing the convolution dimension (when the kernel size in that dimension matches to the number of channels), then in practice the convolution *operation* implemented in a CNN layer of a particular dimensionality can be one dimension size higher. E.g. a layer class labelled "Conv1D" will perform a 2D convolution operation, with the added dimension size matching exactly to the number of input channels. However, conceptually it makes sense to view this as a sum of lower-dimension convolutions, because of the need to exactly match the dimension size. This extra dimension is seen as a convenience for calculation, and not part of the definition.
  
  
  ---
  
  ### Answer by username_3 on 2021-10-01. Score: 1
  
  Kernel dimensionality and presence of filters decides the dimension of convolution operator. N Dimensional Convolutions have N dimensional kernels. For example, from Keras Documentation on [2 Dimensional Convolutions](https://keras.io/api/layers/convolution_layers/convolution2d/):
  
  > kernel\_size: An integer or tuple/list of 2 integers, specifying the height and width of the 2D convolution window. Can be a single integer to specify the same value for all spatial dimensions.
  
  If you have more than one filter in the layer, that also adds another dimension to the layer. So, we can say that a 2D convolutional layer is in general 3 dimensional, where 3rd dimension is the number of filters: (k,k,F). For the special case of a single filter, F=1 and we can treat it as 2 dimensional.
  
  > **Comments:**
  > **username_0**: Please check [this](https://ai.stackexchange.com/a/29977/18758) answer  
  > **username_3**: @hanugm Yes, that is the case in the first layer for a color image, a separate kernel used for each color. Still, each kernel is 2 dimensional. And if we have an additional convolutional layer with 16 filter, we still have 2 dimensional kernels, but now 16 of them.  
  > **username_2**: Worth noting that for efficiency purposes a Conv2D layer is very likely running a 3D convolution operation when feeding forward. This is used to sum over the input channels. However, it is still useful to think of this as conceptually a sum of 2D convolutions when looking from the perspective of signal processing from the input.  
  
  ---
  
  