Answers.Body: <p><code>Model 1</code> and <code>Model 2</code> are very basic CNNs, I would say <a href="https://en.wikipedia.org/wiki/LeNet" rel="nofollow noreferrer">LeNet</a>-style but with max pooling (<code>MP</code>) instead of average pooling. Instead, <code>Model 3</code> resembles a very small <a href="https://arxiv.org/pdf/1409.1556" rel="nofollow noreferrer">VGG</a> net.</p>
  <p>Then regarding parameter count, if <code>c</code> means kernel size (assuming also <code>padding=same</code> otherwise MP won't work at some point) then:</p>
  <ul>
  <li><strong>Model 1</strong>:</li>
  </ul>
  <pre><code> Layer (type)                Output Shape              Param #
  =================================================================
   input_1 (InputLayer)        [(None, 32, 32, 3)]       0
  
   conv2d_12 (Conv2D)          (None, 32, 32, 12)        912
  
   max_pooling2d_4 (MaxPoolin  (None, 16, 16, 12)        0
   g2D)
  
   conv2d_13 (Conv2D)          (None, 16, 16, 64)        19264
  
   max_pooling2d_5 (MaxPoolin  (None, 8, 8, 64)          0
   g2D)
  
   flatten_4 (Flatten)         (None, 4096)              0
  
   dense_6 (Dense)             (None, 10)                40970
  
  =================================================================
  Total params: 61146 (238.85 KB)
  Trainable params: 61146 (238.85 KB)
  Non-trainable params: 0 (0.00 Byte)
  _________________________________________________________________
  </code></pre>
  <ul>
  <li><strong>Model 2:</strong></li>
  </ul>
  <pre><code>_________________________________________________________________
   Layer (type)                Output Shape              Param #
  =================================================================
   input_1 (InputLayer)        [(None, 32, 32, 3)]       0
  
   conv2d_14 (Conv2D)          (None, 32, 32, 32)        2432
  
   max_pooling2d_6 (MaxPoolin  (None, 16, 16, 32)        0
   g2D)
  
   conv2d_15 (Conv2D)          (None, 16, 16, 64)        51264
  
   max_pooling2d_7 (MaxPoolin  (None, 8, 8, 64)          0
   g2D)
  
   conv2d_16 (Conv2D)          (None, 8, 8, 32)          51232
  
   max_pooling2d_8 (MaxPoolin  (None, 4, 4, 32)          0
   g2D)
  
   flatten_5 (Flatten)         (None, 512)               0
  
   dense_8 (Dense)             (None, 100)               51300
  
   dense_9 (Dense)             (None, 10)                1010
  
  =================================================================
  Total params: 157238 (614.21 KB)
  Trainable params: 157238 (614.21 KB)
  Non-trainable params: 0 (0.00 Byte)
  _________________________________________________________________
  </code></pre>
  <ul>
  <li><strong>Model 3:</strong></li>
  </ul>
  <pre><code>_________________________________________________________________
   Layer (type)                Output Shape              Param #
  =================================================================
   input_1 (InputLayer)        [(None, 32, 32, 3)]       0
  
   conv2d_17 (Conv2D)          (None, 32, 32, 32)        896
  
   conv2d_18 (Conv2D)          (None, 32, 32, 32)        9248
  
   max_pooling2d_9 (MaxPoolin  (None, 10, 10, 32)        0
   g2D)
  
   conv2d_19 (Conv2D)          (None, 10, 10, 64)        18496
  
   conv2d_20 (Conv2D)          (None, 10, 10, 32)        18464
  
   max_pooling2d_10 (MaxPooli  (None, 3, 3, 32)          0
   ng2D)
  
   flatten_6 (Flatten)         (None, 288)               0
  
   dense_10 (Dense)            (None, 512)               147968
  
   dense_11 (Dense)            (None, 10)                5130
  
  =================================================================
  Total params: 200202 (782.04 KB)
  Trainable params: 200202 (782.04 KB)
  Non-trainable params: 0 (0.00 Byte)
  _________________________________________________________________
  </code></pre>
  <p>I think the architectures should be close enough to what you're showing.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-08-22T20:39:09.893Z
Answers.Text: What is the max pooling in the last model doing?
Answers.UserDisplayName: Gaslight Deceive Subvert
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-08-23T17:21:19.530Z
Answers.Text: @GaslightDeceiveSubvert In the last model you have `MP(p3s2)` which I interpret as `pool_size=3` (or `kernel_size=3`) with `stride=2`: this is usually called *overlapping max-pooling* because the sliding window is larger than the strides, and so some pixels are considered multiple times when downsampling.
Answers.UserDisplayName: Luca Anzalone
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-08-22T17:35:27.787Z
Answers.LastActivityDate: 2024-08-22T17:35:27.787Z
Answers.UserDisplayName: Luca Anzalone
Body: <p>I'm reading a paper about three CNNs, described in a terse way that I cannot decipher (usually the case for space-efficient papers). Likely, this is due to my ignorance and I wonder if anyone can apply <em>educated guesses</em> to calculate the <em>number of parameters per layer</em> of these CNNs:</p>
  <p><a href="https://i.sstatic.net/WKBJH4wX.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/WKBJH4wX.png" alt="Architecture of three CNNs" /></a></p>
  <p>The input to the CNNs are 32x32x3. Bonus points if someone can connect the models to standard architectures. Model 2 is a bit like LeNet but not exactly.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-09-01T13:21:09.317Z
Comments.Text: I think it would be useful for others to specify the name of the paper :)
Comments.UserDisplayName: Luca Anzalone
Tags: <convolutional-neural-networks><notation>
Title: Deciphering terse CNN specifications
UserDisplayName: Gaslight Deceive Subvert
ThreadText: # Deciphering terse CNN specifications
  
  (Asked by: username_0 on 2024-08-22)
  
  I'm reading a paper about three CNNs, described in a terse way that I cannot decipher (usually the case for space-efficient papers). Likely, this is due to my ignorance and I wonder if anyone can apply *educated guesses* to calculate the *number of parameters per layer* of these CNNs:
  
  [![Architecture of three CNNs](https://i.sstatic.net/WKBJH4wX.png)](https://i.sstatic.net/WKBJH4wX.png)
  
  The input to the CNNs are 32x32x3. Bonus points if someone can connect the models to standard architectures. Model 2 is a bit like LeNet but not exactly.
  
  **Tags:** convolutional-neural-networks, notation
  > **Comments:**
  > **username_1**: I think it would be useful for others to specify the name of the paper :)  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-08-22. Score: 2 | Accepted answer
  
  Model 1 and Model 2 are very basic CNNs, I would say LeNet-style but with max pooling (MP) instead of average pooling. Instead, Model 3 resembles a very small VGG net.
  Then regarding parameter count, if c means kernel size (assuming also padding=same otherwise MP won't work at some point) then:
  
  Model 1:
  
   Layer (type)                Output Shape              Param #
  =================================================================
   input_1 (InputLayer)        [(None, 32, 32, 3)]       0
  
   conv2d_12 (Conv2D)          (None, 32, 32, 12)        912
  
   max_pooling2d_4 (MaxPoolin  (None, 16, 16, 12)        0
   g2D)
  
   conv2d_13 (Conv2D)          (None, 16, 16, 64)        19264
  
   max_pooling2d_5 (MaxPoolin  (None, 8, 8, 64)          0
   g2D)
  
   flatten_4 (Flatten)         (None, 4096)              0
  
   dense_6 (Dense)             (None, 10)                40970
  
  =================================================================
  Total params: 61146 (238.85 KB)
  Trainable params: 61146 (238.85 KB)
  Non-trainable params: 0 (0.00 Byte)
  _________________________________________________________________
  
  
  Model 2:
  
  _________________________________________________________________
   Layer (type)                Output Shape              Param #
  =================================================================
   input_1 (InputLayer)        [(None, 32, 32, 3)]       0
  
   conv2d_14 (Conv2D)          (None, 32, 32, 32)        2432
  
   max_pooling2d_6 (MaxPoolin  (None, 16, 16, 32)        0
   g2D)
  
   conv2d_15 (Conv2D)          (None, 16, 16, 64)        51264
  
   max_pooling2d_7 (MaxPoolin  (None, 8, 8, 64)          0
   g2D)
  
   conv2d_16 (Conv2D)          (None, 8, 8, 32)          51232
  
   max_pooling2d_8 (MaxPoolin  (None, 4, 4, 32)          0
   g2D)
  
   flatten_5 (Flatten)         (None, 512)               0
  
   dense_8 (Dense)             (None, 100)               51300
  
   dense_9 (Dense)             (None, 10)                1010
  
  =================================================================
  Total params: 157238 (614.21 KB)
  Trainable params: 157238 (614.21 KB)
  Non-trainable params: 0 (0.00 Byte)
  _________________________________________________________________
  
  
  Model 3:
  
  _________________________________________________________________
   Layer (type)                Output Shape              Param #
  =================================================================
   input_1 (InputLayer)        [(None, 32, 32, 3)]       0
  
   conv2d_17 (Conv2D)          (None, 32, 32, 32)        896
  
   conv2d_18 (Conv2D)          (None, 32, 32, 32)        9248
  
   max_pooling2d_9 (MaxPoolin  (None, 10, 10, 32)        0
   g2D)
  
   conv2d_19 (Conv2D)          (None, 10, 10, 64)        18496
  
   conv2d_20 (Conv2D)          (None, 10, 10, 32)        18464
  
   max_pooling2d_10 (MaxPooli  (None, 3, 3, 32)          0
   ng2D)
  
   flatten_6 (Flatten)         (None, 288)               0
  
   dense_10 (Dense)            (None, 512)               147968
  
   dense_11 (Dense)            (None, 10)                5130
  
  =================================================================
  Total params: 200202 (782.04 KB)
  Trainable params: 200202 (782.04 KB)
  Non-trainable params: 0 (0.00 Byte)
  _________________________________________________________________
  
  I think the architectures should be close enough to what you're showing.
  
  > **Comments:**
  > **username_0**: What is the max pooling in the last model doing?  
  > **username_1**: @GaslightDeceiveSubvert In the last model you have `MP(p3s2)` which I interpret as `pool_size=3` (or `kernel_size=3`) with `stride=2`: this is usually called *overlapping max-pooling* because the sliding window is larger than the strides, and so some pixels are considered multiple times when downsampling.  
  
  ---
  
  