Answers.Body: <p>3D convolutions should be used when you want to extract spatial features from your input on 3 dimensions. For computer vision, they are typically used on <strong>volumetric images</strong>, which are 3D.</p>
  <p>Some examples are <a href="http://graphics.stanford.edu/projects/3dcnn/" rel="nofollow noreferrer">classifying 3D rendered images</a> and <a href="https://arxiv.org/pdf/1606.06650.pdf" rel="nofollow noreferrer">medical image segmentation</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-05T00:38:13.260Z
Answers.LastActivityDate: 2021-12-18T12:57:57.183Z
Answers.LastEditDate: 2021-12-18T12:57:57.183Z
Answers.UserDisplayName: Jerome
Answers.Body: <p>3D convolutions are used when you want to extract features in 3 dimensions or establish a relationship between 3 dimensions.</p>
  <p>Essentially, it's the same as 2D convolutions, but the kernel movement is now 3-dimensional, causing a better capture of dependencies within the 3 dimensions and a difference in output dimensions post convolution.</p>
  <p>The kernel of the 3d convolution will move in 3-dimensions if the kernel's depth is lesser than the feature map's depth.</p>
  <p><img src="https://miro.medium.com/max/700/1*wUVVgZnzBwYKgQyTBK_5sg.png" alt="this" /></p>
  <p>On the other hand, 2-D convolutions on 3-D data mean that the kernel will traverse in 2-D only. This happens when the feature map's depth is the same as the kernel's depth (channels).</p>
  <p><img src="https://miro.medium.com/max/700/1*K_OsaLGJ7I8cGlPvvWYJdg.png" alt="here" /></p>
  <p>Some use cases for better understanding are</p>
  <ul>
  <li><p>MRI scans where the relationship between a stack of images is to be understood;</p>
  </li>
  <li><p>low-level feature extractor for spatio-temporal data, like videos for gesture recognition, weather forecasting, etc. (3-D CNN's are used as low level feature extractors only over multiple short intervals, as 3D CNN's fail to capture long term spatio-temporal dependencies - for more on that check out <a href="https://papers.nips.cc/paper/5955-convolutional-lstm-network-a-machine-learning-approach-for-precipitation-nowcasting.pdf" rel="noreferrer">ConvLSTM</a> or an alternate perspective <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6387284/" rel="noreferrer">here</a>.)</p>
  </li>
  </ul>
  <p>Most CNN models that learn from video data almost always have 3D CNN as a low level feature extractor.</p>
  <p>In the example you have mentioned above regarding the number 5 - 2D convolutions would probably perform better, as you're treating every channel intensity as an aggregate of the information it holds, meaning the learning would almost be the same as it would on a black and white image. Using 3D convolution for this, on the other hand, would cause learning of relationships between the channels which do not exist in this case! (Also 3D convolutions on an image with depth 3 would require a very uncommon kernel to be used, especially for the use case)</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-02-05T04:54:58.003Z
Answers.Text: How did you create those 3d conv figures?
Answers.UserDisplayName: nikhilweee
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-05T07:43:25.680Z
Answers.LastActivityDate: 2021-12-18T12:57:04.917Z
Answers.LastEditDate: 2021-12-18T12:57:04.917Z
Answers.UserDisplayName: ashenoy
Body: <p>I am new to convolutional neural networks, and I am learning 3D convolution. What I could understand is that 2D convolution gives us relationships between low-level features in the X-Y dimension, while the 3D convolution helps detect low-level features and relationships between them in all the 3 dimensions.</p>
  
  <p>Consider a CNN employing 2D convolutional layers to recognize handwritten digits. If a digit, say 5, was written in different colors:</p>
  
  <p><a href="https://i.sstatic.net/3Ajlf.jpg" rel="noreferrer"><img src="https://i.sstatic.net/3Ajlf.jpg" alt="enter image description here"></a></p>
  
  <p>Would a strictly 2D CNN perform poorly (since they belong to different channels in the z-dimension)?</p>
  
  <p>Also, are there practical well-known neural nets that employ 3D convolution?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-31T06:37:48.457Z
Comments.Text: 3D convolutions are commonly used for processing 3D images such as MRI scans.
Comments.UserDisplayName: Yashas
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-31T06:53:15.803Z
Comments.Text: Are there any publications on 3D Conv architectures?
Comments.UserDisplayName: Shobhit Verma
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-08-07T11:50:49.197Z
Comments.Text: @Shobhit given the answer by ashenoy, is there some part of your question that has not been answered yet?
Comments.UserDisplayName: mshlis
Tags: <convolutional-neural-networks><reference-request><convolution><convolutional-layers><3d-convolution>
Title: When should I use 3D convolutions?
UserDisplayName: Shobhit Verma
ThreadText: # When should I use 3D convolutions?
  
  (Asked by: username_0 on 2019-07-31)
  
  I am new to convolutional neural networks, and I am learning 3D convolution. What I could understand is that 2D convolution gives us relationships between low-level features in the X-Y dimension, while the 3D convolution helps detect low-level features and relationships between them in all the 3 dimensions.
  
  Consider a CNN employing 2D convolutional layers to recognize handwritten digits. If a digit, say 5, was written in different colors:
  
  [![enter image description here](https://i.sstatic.net/3Ajlf.jpg)](https://i.sstatic.net/3Ajlf.jpg)
  
  Would a strictly 2D CNN perform poorly (since they belong to different channels in the z-dimension)?
  
  Also, are there practical well-known neural nets that employ 3D convolution?
  
  **Tags:** convolutional-neural-networks, reference-request, convolution, convolutional-layers, 3d-convolution
  
  > **Comments:**
  > **username_1**: 3D convolutions are commonly used for processing 3D images such as MRI scans.  
  > **username_0**: Are there any publications on 3D Conv architectures?  
  > **username_2**: @username_0 given the answer by username_4, is there some part of your question that has not been answered yet?  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_4 on 2019-08-05. Score: 19 | Accepted answer
  
  3D convolutions are used when you want to extract features in 3 dimensions or establish a relationship between 3 dimensions.
  
  Essentially, it's the same as 2D convolutions, but the kernel movement is now 3-dimensional, causing a better capture of dependencies within the 3 dimensions and a difference in output dimensions post convolution.
  
  The kernel of the 3d convolution will move in 3-dimensions if the kernel's depth is lesser than the feature map's depth.
  
  ![this](https://miro.medium.com/max/700/1*wUVVgZnzBwYKgQyTBK_5sg.png)
  
  On the other hand, 2-D convolutions on 3-D data mean that the kernel will traverse in 2-D only. This happens when the feature map's depth is the same as the kernel's depth (channels).
  
  ![here](https://miro.medium.com/max/700/1*K_OsaLGJ7I8cGlPvvWYJdg.png)
  
  Some use cases for better understanding are
  
  * MRI scans where the relationship between a stack of images is to be understood;
  * low-level feature extractor for spatio-temporal data, like videos for gesture recognition, weather forecasting, etc. (3-D CNN's are used as low level feature extractors only over multiple short intervals, as 3D CNN's fail to capture long term spatio-temporal dependencies - for more on that check out [ConvLSTM](https://papers.nips.cc/paper/5955-convolutional-lstm-network-a-machine-learning-approach-for-precipitation-nowcasting.pdf) or an alternate perspective [here](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6387284/).)
  
  Most CNN models that learn from video data almost always have 3D CNN as a low level feature extractor.
  
  In the example you have mentioned above regarding the number 5 - 2D convolutions would probably perform better, as you're treating every channel intensity as an aggregate of the information it holds, meaning the learning would almost be the same as it would on a black and white image. Using 3D convolution for this, on the other hand, would cause learning of relationships between the channels which do not exist in this case! (Also 3D convolutions on an image with depth 3 would require a very uncommon kernel to be used, especially for the use case)
  
  > **Comments:**
  > **username_5**: How did you create those 3d conv figures?  
  
  ---
  
  ### Answer by username_3 on 2019-08-05. Score: 6
  
  3D convolutions should be used when you want to extract spatial features from your input on 3 dimensions. For computer vision, they are typically used on **volumetric images**, which are 3D.
  
  Some examples are [classifying 3D rendered images](http://graphics.stanford.edu/projects/3dcnn/) and [medical image segmentation](https://arxiv.org/pdf/1606.06650.pdf).
  
  
  ---
  
  