Answers.Body: <p>This is largely a matter of semantics I think. After the avg pool layer, the feature map becomes <span class="math-container">$1\times 1\times 1280$</span> as indicated on the table, so when you apply a pointwise convolution (a <span class="math-container">$1 \times 1$</span> conv), it essentially serves as a fully-connected layer operating on the channel dimension since there are no spaital dimensions to work with anymore.</p>
  <p>If you want, you can think of the last layer as a dense layer with weights shaped <span class="math-container">$1280 \times k$</span>, where the <span class="math-container">$1280$</span> input channels are interpreted as input features to the dense layer, and where <span class="math-container">$k$</span> is the number of classes.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-16T22:47:34.527Z
Answers.Text: I was having a hard time understanding the purpose of that final '1x1 conv layer' but your answer really made it clear for me! Thanks for that! I guess I didn't understand the semantics behind the usage of '1x1 conv layer' instead of a simple 'dense layer'. Is this a common practice when writting papers? Thank you again!
Answers.UserDisplayName: Blue Ross
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-17T05:32:12.747Z
Answers.Text: @BlueRoss I don't know if this convention is common, but I've seen multiple conventions (e.g. 1x1 conv, dense, fc, etc.) being used to describe the classification layer of a convnet. You may want to look at other major convnet papers (e.g. ResNet, other mobilenet versions) to see what they use
Answers.UserDisplayName: PeaBrane
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-16T17:08:37.830Z
Answers.LastActivityDate: 2022-10-16T17:08:37.830Z
Answers.UserDisplayName: PeaBrane
Body: <p>So I was studying <a href="https://arxiv.org/abs/1801.04381" rel="nofollow noreferrer">MobileNetV2</a> architecture and came across this table from the original paper that represents its architecture:</p>
  <blockquote>
  <p><a href="https://i.sstatic.net/1RAkv.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/1RAkv.png" alt="Table 2: MobileNetV2 : Each line describes a sequence of 1 or more identical (modulo stride) layers, repeated n times. All layers in the same sequence have the same number c of output channels. The first layer of each sequence has a stride s and all others use stride 1. All spatial convolutions use 3 × 3 kernels. The expansion factor t is always applied to the input size as described in Table 1." /></a>
  Table Description: &quot;Table 2: MobileNetV2 : Each line describes a sequence of 1 or more identical (modulo stride) layers, repeated n times. All layers in the same sequence have the same number c of output channels. The first layer of each sequence has a stride s and all others use stride 1. All spatial convolutions use 3 × 3 kernels. The expansion factor t is always applied to the input size as described in Table 1.&quot;</p>
  </blockquote>
  <p>However I don't really understand why the last pointwise convolution (conv2d 1x1) is applied after the average pooling.
  Shouldn't the output from the avgpool go to a dense layer so that it can then perform its predictions?</p>
  <p>What is the point of performing convolution, more precisely pointwise conv. after pooling? Also, why is <em>k</em> not a fixed number of kernels/filters, like the other layers?</p>
  <p>Thanks in advance!</p>
  
Tags: <deep-learning><deep-neural-networks><mobile-net-v2>
Title: MobileNetV2 - Some particularities
UserDisplayName: Blue Ross
ThreadText: # MobileNetV2 - Some particularities
  
  (Asked by: username_0 on 2022-10-15)
  
  So I was studying [MobileNetV2](https://arxiv.org/abs/1801.04381) architecture and came across this table from the original paper that represents its architecture:
  
  > [![Table 2: MobileNetV2 : Each line describes a sequence of 1 or more identical (modulo stride) layers, repeated n times. All layers in the same sequence have the same number c of output channels. The first layer of each sequence has a stride s and all others use stride 1. All spatial convolutions use 3 × 3 kernels. The expansion factor t is always applied to the input size as described in Table 1.](https://i.sstatic.net/1RAkv.png)](https://i.sstatic.net/1RAkv.png)
  > Table Description: "Table 2: MobileNetV2 : Each line describes a sequence of 1 or more identical (modulo stride) layers, repeated n times. All layers in the same sequence have the same number c of output channels. The first layer of each sequence has a stride s and all others use stride 1. All spatial convolutions use 3 × 3 kernels. The expansion factor t is always applied to the input size as described in Table 1."
  
  However I don't really understand why the last pointwise convolution (conv2d 1x1) is applied after the average pooling.
  Shouldn't the output from the avgpool go to a dense layer so that it can then perform its predictions?
  
  What is the point of performing convolution, more precisely pointwise conv. after pooling? Also, why is *k* not a fixed number of kernels/filters, like the other layers?
  
  Thanks in advance!
  
  **Tags:** deep-learning, deep-neural-networks, mobile-net-v2
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-10-16. Score: 1 | Accepted answer
  
  This is largely a matter of semantics I think. After the avg pool layer, the feature map becomes $1\times 1\times 1280$ as indicated on the table, so when you apply a pointwise convolution (a $1 \times 1$ conv), it essentially serves as a fully-connected layer operating on the channel dimension since there are no spaital dimensions to work with anymore.
  
  If you want, you can think of the last layer as a dense layer with weights shaped $1280 \times k$, where the $1280$ input channels are interpreted as input features to the dense layer, and where $k$ is the number of classes.
  
  > **Comments:**
  > **username_0**: I was having a hard time understanding the purpose of that final '1x1 conv layer' but your answer really made it clear for me! Thanks for that! I guess I didn't understand the semantics behind the usage of '1x1 conv layer' instead of a simple 'dense layer'. Is this a common practice when writting papers? Thank you again!  
  > **username_1**: @username_0 I don't know if this convention is common, but I've seen multiple conventions (e.g. 1x1 conv, dense, fc, etc.) being used to describe the classification layer of a convnet. You may want to look at other major convnet papers (e.g. ResNet, other mobilenet versions) to see what they use  
  
  ---
  
  