Answers.Body: <p>By reading the <a href="https://arxiv.org/pdf/1711.11248v3.pdf" rel="nofollow noreferrer">paper</a> that proposed the idea of (2+1)Convolutions, it seems that the <code>Conv2Plus1D</code> implementation of the tutorial is <em>partially</em> incorrect.</p>
  <p>By citing the paper (section 3.5):</p>
  <blockquote>
  <p>[...] full 3D convolutions may be more conveniently approximated by a 2D convolution followed by a 1D convolution, decomposing spatial and temporal modeling into two separate steps.</p>
  </blockquote>
  <p>So they assume a 3D conv, with an <span class="math-container">$N\times t\times d\times d$</span> kernel (where <span class="math-container">$t$</span> is the timesteps, and <span class="math-container">$d$</span> the depth), can be &quot;factorized&quot; into space and time components by two convolutions: one is <span class="math-container">$N\times 1\times d \times d$</span> with <span class="math-container">$M$</span> filters, and the other has a <span class="math-container">$M\times t\times \times 1$</span> kernel with <span class="math-container">$N$</span> filters. The former is the spatial convolution, and the latter is the temporal one.</p>
  <p>Now:</p>
  <blockquote>
  <p>The <strong>hyperparameter</strong> <span class="math-container">$M$</span> determines the dimensionality of the intermediate subspace.</p>
  </blockquote>
  <p>They provide a formula to compute <span class="math-container">$M$</span>:
  <span class="math-container">$$M_i = \frac{td^2 N_{i-1}N_i}{d^2 N_{i-1} + tN_i},$$</span>
  where <span class="math-container">$i$</span> is the layer index. Basically, with such a formula they choose <span class="math-container">$M$</span> so that:</p>
  <blockquote>
  <p>[...] the number of parameters in the (2+1)D block is <strong>approximately equal</strong> to that implementing full 3D convolution.</p>
  </blockquote>
  <p>In short, the first conv in <code>Conv2Plus1D</code> should have <span class="math-container">$M_i$</span> filters, and the second conv <span class="math-container">$N_i$</span> filters (i.e., the number you would use for the regular 3D conv.). Indeed, can be that <span class="math-container">$M_i = N_i$</span> as in the tutorial but that won't follow the paper. Now, if you compute <span class="math-container">$M_i$</span> according to the eq. you'll get approximately the same overall number of parameters compared to a full 3D conv. But you can set it as you wish, thus also reducing the #parameters if you want to do so.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-06-24T16:44:26.660Z
Answers.Text: Appreciate the time. I also thought the same thing as I read the paper but because the tutorial was on TensorFlow's own website, I got the sense that I might be missing something. Anyways, thanks for the help.
Answers.UserDisplayName: Ali Khalili
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-06-24T12:23:04.190Z
Answers.LastActivityDate: 2023-07-18T00:16:47.360Z
Answers.LastEditDate: 2023-07-18T00:16:47.360Z
Answers.UserDisplayName: Luca Anzalone
Body: <p>Based on <a href="https://www.tensorflow.org/tutorials/video/video_classification" rel="nofollow noreferrer">this</a> tutorial, and the</p>
  <blockquote>
  <p>An advantage of this approach is that factorizing the convolutions into spatial and temporal dimensions saves parameters.</p>
  </blockquote>
  <p>statement, the Conv2Plus1D must have fewer parameters than a Conv3D layer. I compared the number of parameters between a Conv2Plus1D and a Conv3D layer and the Conv3D layer had much fewer parameters (around 1/3 of the Conv2Plus1D) than a Conv3D layer.</p>
  <p>Am I missing something? Or am I misunderstanding something?
  Please clarify if you can.</p>
  
Tags: <convolutional-neural-networks><convolution>
Title: How does Conv2Plus1D reduces the number of paramateres?
UserDisplayName: Ali Khalili
ThreadText: # How does Conv2Plus1D reduces the number of paramateres?
  
  (Asked by: username_0 on 2023-06-24)
  
  Based on [this](https://www.tensorflow.org/tutorials/video/video_classification) tutorial, and the
  
  > An advantage of this approach is that factorizing the convolutions into spatial and temporal dimensions saves parameters.
  
  statement, the Conv2Plus1D must have fewer parameters than a Conv3D layer. I compared the number of parameters between a Conv2Plus1D and a Conv3D layer and the Conv3D layer had much fewer parameters (around 1/3 of the Conv2Plus1D) than a Conv3D layer.
  
  Am I missing something? Or am I misunderstanding something?
  Please clarify if you can.
  
  **Tags:** convolutional-neural-networks, convolution
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-06-24. Score: 1 | Accepted answer
  
  By reading the [paper](https://arxiv.org/pdf/1711.11248v3.pdf) that proposed the idea of (2+1)Convolutions, it seems that the `Conv2Plus1D` implementation of the tutorial is *partially* incorrect.
  
  By citing the paper (section 3.5):
  
  > [...] full 3D convolutions may be more conveniently approximated by a 2D convolution followed by a 1D convolution, decomposing spatial and temporal modeling into two separate steps.
  
  So they assume a 3D conv, with an $N\times t\times d\times d$ kernel (where $t$ is the timesteps, and $d$ the depth), can be "factorized" into space and time components by two convolutions: one is $N\times 1\times d \times d$ with $M$ filters, and the other has a $M\times t\times \times 1$ kernel with $N$ filters. The former is the spatial convolution, and the latter is the temporal one.
  
  Now:
  
  > The **hyperparameter** $M$ determines the dimensionality of the intermediate subspace.
  
  They provide a formula to compute $M$:
  $$M\_i = \frac{td^2 N\_{i-1}N\_i}{d^2 N\_{i-1} + tN\_i},$$
  where $i$ is the layer index. Basically, with such a formula they choose $M$ so that:
  
  > [...] the number of parameters in the (2+1)D block is **approximately equal** to that implementing full 3D convolution.
  
  In short, the first conv in `Conv2Plus1D` should have $M\_i$ filters, and the second conv $N\_i$ filters (i.e., the number you would use for the regular 3D conv.). Indeed, can be that $M\_i = N\_i$ as in the tutorial but that won't follow the paper. Now, if you compute $M\_i$ according to the eq. you'll get approximately the same overall number of parameters compared to a full 3D conv. But you can set it as you wish, thus also reducing the #parameters if you want to do so.
  
  > **Comments:**
  > **username_0**: Appreciate the time. I also thought the same thing as I read the paper but because the tutorial was on TensorFlow's own website, I got the sense that I might be missing something. Anyways, thanks for the help.  
  
  ---
  
  