Answers.Body: <p>1x1 convolutions is a very simple trick generalized by the Inception module published by Google in 2014 in the famous paper <a href="https://arxiv.org/abs/1409.4842" rel="nofollow noreferrer">Going Deeper with Convolutions</a>.</p>
  <p>The most common use case is to modify the output channels of the input feature maps. This is mostly used when you have a net architecture with multiple branches that needs to be aggregated into one result (such as in your image or such as in the Inception module).</p>
  <p>Here is an example:</p>
  <ul>
  <li>Input Feature map: <span class="math-container">$F_{in}: B \times H \times W \times C_{in}$</span></li>
  <li>Convolution: <span class="math-container">$W: 1 \times 1 \times C_{out}$</span></li>
  <li>Output Feature map: <span class="math-container">$F_{out}: B \times H \times W \times C_{out}$</span></li>
  </ul>
  <p>In plain english: <strong>1x1 convolutions modify the output feature maps channels <span class="math-container">$C$</span> without altering the resolution <span class="math-container">$H \times W$</span>.</strong></p>
  <p>When is this useful? For when you need to expand or shrink the feature maps channels. Examples: Inception, SE Blocks, Bottlenecks Blocks, Detectors Head (RetinaHead)...</p>
  <p><a href="https://medium.com/analytics-vidhya/talented-mr-1x1-comprehensive-look-at-1x1-convolution-in-deep-learning-f6b355825578" rel="nofollow noreferrer">Here is</a> a more in depth article</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-03-21T12:29:16.560Z
Answers.Text: many thanks! So is this the same as 'point convolution' ?
Answers.UserDisplayName: James Arten
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-03-21T12:47:13.617Z
Answers.Text: Yes, exactly. In some papers they say point convolution or fully connected layers, but in the code they are implemented with 1x1 conv
Answers.UserDisplayName: JVGD
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-03-21T12:23:48.250Z
Answers.LastActivityDate: 2022-03-21T12:23:48.250Z
Answers.UserDisplayName: JVGD
Body: <p>I'm reading this about Self-Attention GANs : <a href="https://sthalles.github.io/advanced_gans/" rel="nofollow noreferrer">https://sthalles.github.io/advanced_gans/</a></p>
  <p>I'm trying to better frame what is intended about <code>1x1</code>convolution for input images. Does this simply mean that we produce feature maps by multiplying a scalar filter to each single pixel of the image? It's not clear since in the reported picture the <code>1x1conv</code> part resembles a vector.</p>
  <p><a href="https://i.sstatic.net/fpmsI.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/fpmsI.png" alt="enter image description here" /></a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-03-21T11:58:31.100Z
Comments.Text: I had not read this work, but I am quite curious since it seems to me that this would just create three sets of layers that are just scaled representations of each other.  Though, that could be the purpose given the different intentions of $f(x), g(x), and h(x)$ in self attention models.
Comments.UserDisplayName: David Hoelzer
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-03-21T12:13:51.893Z
Comments.Text: Anyway I think that my question is kinda trivial: saying we have feature maps of dimension $(N, K,K)$ where $N$ is the batch size, then convolving them with $1 \times 1$ filters (single-channel) will return transformed tensors with the same dimensions $(N,K,K)$ and the image is a bit misleading from my perspective. Is this correct? Also, convolving with $1 \times 1$ filters is what it's usually named 'Point convolution' ?
Comments.UserDisplayName: James Arten
Tags: <convolutional-neural-networks><generative-adversarial-networks>
Title: What is intended for 1x1 convolution for input images?
UserDisplayName: James Arten
ThreadText: # What is intended for 1x1 convolution for input images?
  
  (Asked by: username_0 on 2022-03-18)
  
  I'm reading this about Self-Attention GANs : <https://sthalles.github.io/advanced_gans/>
  
  I'm trying to better frame what is intended about `1x1`convolution for input images. Does this simply mean that we produce feature maps by multiplying a scalar filter to each single pixel of the image? It's not clear since in the reported picture the `1x1conv` part resembles a vector.
  
  [![enter image description here](https://i.sstatic.net/fpmsI.png)](https://i.sstatic.net/fpmsI.png)
  
  **Tags:** convolutional-neural-networks, generative-adversarial-networks
  > **Comments:**
  > **username_1**: I had not read this work, but I am quite curious since it seems to me that this would just create three sets of layers that are just scaled representations of each other.  Though, that could be the purpose given the different intentions of $f(x), g(x), and h(x)$ in self attention models.  
  > **username_0**: Anyway I think that my question is kinda trivial: saying we have feature maps of dimension $(N, K,K)$ where $N$ is the batch size, then convolving them with $1 \times 1$ filters (single-channel) will return transformed tensors with the same dimensions $(N,K,K)$ and the image is a bit misleading from my perspective. Is this correct? Also, convolving with $1 \times 1$ filters is what it's usually named 'Point convolution' ?  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2022-03-21. Score: 1 | Accepted answer
  
  1x1 convolutions is a very simple trick generalized by the Inception module published by Google in 2014 in the famous paper [Going Deeper with Convolutions](https://arxiv.org/abs/1409.4842).
  
  The most common use case is to modify the output channels of the input feature maps. This is mostly used when you have a net architecture with multiple branches that needs to be aggregated into one result (such as in your image or such as in the Inception module).
  
  Here is an example:
  
  * Input Feature map: $F\_{in}: B \times H \times W \times C\_{in}$
  * Convolution: $W: 1 \times 1 \times C\_{out}$
  * Output Feature map: $F\_{out}: B \times H \times W \times C\_{out}$
  
  In plain english: **1x1 convolutions modify the output feature maps channels $C$ without altering the resolution $H \times W$.**
  
  When is this useful? For when you need to expand or shrink the feature maps channels. Examples: Inception, SE Blocks, Bottlenecks Blocks, Detectors Head (RetinaHead)...
  
  [Here is](https://medium.com/analytics-vidhya/talented-mr-1x1-comprehensive-look-at-1x1-convolution-in-deep-learning-f6b355825578) a more in depth article
  
  > **Comments:**
  > **username_0**: many thanks! So is this the same as 'point convolution' ?  
  > **username_2**: Yes, exactly. In some papers they say point convolution or fully connected layers, but in the code they are implemented with 1x1 conv  
  
  ---
  
  