Answers.Body: <p>Looking at it from the perspective of input to output in that fashion is probably not the best. So lets start with our goal and how these ND convolutions accomplish that (Note these are in my own words, and may not be best stated).  </p>
  
  <p><strong>Assumption:</strong> There exists highly correlative local associations  </p>
  
  <p><strong>Goal:</strong> Have a linear model that takes advantage of these local associations  </p>
  
  <p><strong>Solution:</strong> The ND Convolution  </p>
  
  <p><strong>Explanation:</strong> ND convolutions take advantage of our locality assumption by connecting only local nodes/neurons. The fact that its a sliding window allows us to learn filters for any location along with ones that can be reusable.  </p>
  
  <p><strong>Your Question:</strong>  <em>Where does the N in ND convolution matter?:</em><br>
  When using an ND convolution we are working off the assumption that there exists this locality in <strong>N</strong> dimensions and nothing more. So we connect all other components of the input in a dense matter because we have no assumptions to work on in this space. So now going to the shapes you mentioned such as the input and output of the <strong>2</strong>D convolution. We are convolving a filter of size <span class="math-container">$(C_{in}, k_h, k_w)$</span> with a <span class="math-container">$(C_{in},H,W)$</span> activation shaped map (the <span class="math-container">$N$</span> just refers to the number of activation maps, and there is no association between them). We use <span class="math-container">$C_{in}$</span> channels on the kernel because we are not making any assumptions about locality between channels.  On the other hand in a <strong>3</strong>D convolution we make locality assumption is 3 dimensions, so our kernel will be (<span class="math-container">$C_{in}, k_h, k_w, k_r$</span>). These kernel sizes are actually determined by the input size, the amount of dimensions of the kernel will actually match the inputs (minus the batch) because it needs to densely match each one.  </p>
  
  <p>You may be thinking now, that in torch this is not the case: This is because its rare to want a 2D Convolution with an inputs that's different than the one you mention, so they only implemented it for a singular shape. I hope this clears up the convolutions and helps you understand not just for 2 and 3 dimensional convolutions, but for all N. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-13T19:20:27.837Z
Answers.LastActivityDate: 2019-08-13T19:20:27.837Z
Answers.UserDisplayName: mshlis
Body: <p>I was trying to understand the definition of 2d convolutions vs 3d convolutions. I saw the "simplest definition" according to Pytorch and it seems the following:</p>
  
  <ul>
  <li>2d convolutions map <span class="math-container">$(N,C_{in},H,W) \rightarrow (N,C_{out},H_{out},W_{out})$</span></li>
  <li>3d convolutions map <span class="math-container">$(N,C_{in},D,H,W) \rightarrow (N,C_{out},D_{out},H_{out},W_{out})$</span></li>
  </ul>
  
  <p>Which make sense to me. However, what I find confusing is that I would have expected images to be considered 3D tensors but we apply 2D convolutions to them. Why is that? Why is the channel tensor not part of the "definitionality of the images"?</p>
  
  <p>I also asked this question at <a href="https://forums.fast.ai/t/what-is-the-difference-between-2d-vs-3d-convolutions/52495" rel="nofollow noreferrer">https://forums.fast.ai/t/what-is-the-difference-between-2d-vs-3d-convolutions/52495</a>.</p>
  
Tags: <deep-learning><convolutional-neural-networks><computer-vision><comparison>
Title: What is the difference between 2d vs 3d convolutions?
UserDisplayName: Charlie Parker
ThreadText: # What is the difference between 2d vs 3d convolutions?
  
  (Asked by: username_0 on 2019-08-13)
  
  I was trying to understand the definition of 2d convolutions vs 3d convolutions. I saw the "simplest definition" according to Pytorch and it seems the following:
  
  * 2d convolutions map $(N,C\_{in},H,W) \rightarrow (N,C\_{out},H\_{out},W\_{out})$
  * 3d convolutions map $(N,C\_{in},D,H,W) \rightarrow (N,C\_{out},D\_{out},H\_{out},W\_{out})$
  
  Which make sense to me. However, what I find confusing is that I would have expected images to be considered 3D tensors but we apply 2D convolutions to them. Why is that? Why is the channel tensor not part of the "definitionality of the images"?
  
  I also asked this question at <https://forums.fast.ai/t/what-is-the-difference-between-2d-vs-3d-convolutions/52495>.
  
  **Tags:** deep-learning, convolutional-neural-networks, computer-vision, comparison
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-08-13. Score: 1
  
  Looking at it from the perspective of input to output in that fashion is probably not the best. So lets start with our goal and how these ND convolutions accomplish that (Note these are in my own words, and may not be best stated).
  
  **Assumption:** There exists highly correlative local associations
  
  **Goal:** Have a linear model that takes advantage of these local associations
  
  **Solution:** The ND Convolution
  
  **Explanation:** ND convolutions take advantage of our locality assumption by connecting only local nodes/neurons. The fact that its a sliding window allows us to learn filters for any location along with ones that can be reusable.
  
  **Your Question:** *Where does the N in ND convolution matter?:*  
  When using an ND convolution we are working off the assumption that there exists this locality in **N** dimensions and nothing more. So we connect all other components of the input in a dense matter because we have no assumptions to work on in this space. So now going to the shapes you mentioned such as the input and output of the **2**D convolution. We are convolving a filter of size $(C\_{in}, k\_h, k\_w)$ with a $(C\_{in},H,W)$ activation shaped map (the $N$ just refers to the number of activation maps, and there is no association between them). We use $C\_{in}$ channels on the kernel because we are not making any assumptions about locality between channels. On the other hand in a **3**D convolution we make locality assumption is 3 dimensions, so our kernel will be ($C\_{in}, k\_h, k\_w, k\_r$). These kernel sizes are actually determined by the input size, the amount of dimensions of the kernel will actually match the inputs (minus the batch) because it needs to densely match each one.
  
  You may be thinking now, that in torch this is not the case: This is because its rare to want a 2D Convolution with an inputs that's different than the one you mention, so they only implemented it for a singular shape. I hope this clears up the convolutions and helps you understand not just for 2 and 3 dimensional convolutions, but for all N.
  
  
  ---
  
  