Answers.Body: <p>I'm not quite sure what you mean by &quot;combining&quot; these maps, but here is a simple example (in Keras):</p>
  <pre><code>model = keras.models.Sequential([
      layers.InputLayer((res, res, 1)),
      layers.Conv2D(3, 7, activation='sigmoid'),
      layers.Conv2D(3, 7, activation='sigmoid'),
      
      layers.GlobalMaxPooling2D(),
      layers.Dense(1, activation='sigmoid')
  ])
  </code></pre>
  <pre><code>Layer (type)                 Output Shape              Param #   
  =================================================================
  conv2d_199 (Conv2D)          (None, 42, 42, 3)         150       
  _________________________________________________________________
  conv2d_200 (Conv2D)          (None, 36, 36, 3)         444       
  _________________________________________________________________
  global_max_pooling2d_92 (Glo (None, 3)                 0         
  _________________________________________________________________
  dense_102 (Dense)            (None, 1)                 4         
  =================================================================
  Total params: 598
  Trainable params: 598
  Non-trainable params: 0
  </code></pre>
  <p>I used sigmoid activations on the convolutional layers as well (with three &quot;kernels&quot;), since their outputs are easy to visualize as RGB images.</p>
  <p>I trained the network on generated 48x48 grayscale images, each showing either an ellipse or a rectangle. Input images are shown on the top right, and they are normalized so that 25% of the pixels are black and 2% are white.</p>
  <p>The intermediate output is quite hard to interpret, but the second output (before global max pooling) correlates well with the input images, and also with the output target class. The green channel seems to correspond to circles and ellipses (detecting curvature?), while red and blue channels react to linear lines. These correlations are also shown in the lower plot.</p>
  <blockquote>
  <p>Then when we add a second convolutional layer, how can it extract a higher order feature without combining feature maps in some way?</p>
  </blockquote>
  <p>Note that while the first Conv2D layer has weights of shape <code>[7, 7, 1, 3]</code> (ignoring the bias), the seconds one has a shape of <code>[7, 7, 3, 3]</code>. Eg. it sees all three channels of the previous layer simultaneously, meaning three &quot;separate&quot; matrices. So &quot;convolutional layers only apply the filter on <strong>one matrix</strong> at a time&quot; isn't quite true. Instead they apply to <strong>one tensor</strong> at a time, which can be interpreted as stacked matrices.</p>
  <p><a href="https://i.sstatic.net/85VP8.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/85VP8.jpg" alt="network outputs 1" /></a></p>
  <p>Note that the network may learn to detect very different aspects, depending on the initial parameters and specifics of the data. For example in this case the shapes are filled solid, and the network makes the distinction whether there are corners in the image or not. The green channel alone doesn't seem to be good for this classification task.</p>
  <p><a href="https://i.sstatic.net/Db0CW.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Db0CW.jpg" alt="network outputs 2" /></a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-22T11:37:19.687Z
Answers.Text: Thanks so much this definitely cleared up my confusion!
  
  "So "convolutional layers only apply the filter on one matrix at a time" isn't quite true. Instead they apply to one tensor at a time, which can be interpreted as stacked matrices."
  
  Okay, I see the filter/kernel can also be a tensor. Interesting. So would the order of your feature maps matter? It would seem to me that they would then. In this case where there is 3 channels it wouldn't, but if the kernel was smaller, the order of the feature maps would make some difference right?
Answers.UserDisplayName: Brian Przezdziecki
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-22T11:38:22.213Z
Answers.Text: Wait actually, no it doesn't. Because the kernels before are learned. Okay sorry if I'm being confusing, don't worry about it I got it now!
Answers.UserDisplayName: Brian Przezdziecki
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-22T19:30:18.580Z
Answers.Text: Ah, a simpler way to put this is to think of a color image with RGB color channels. The first convolution kernel looks at all of the color channel simultaneously, while seeing just a certain part of the image (depending on the kernel's width and height). And usually the number of channels increases as we add more layers to the network, but the math is still the same. A RGB image can be thought of as three matrices, or a single tensor.
Answers.UserDisplayName: NikoNyrh
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-21T16:15:43.430Z
Answers.LastActivityDate: 2022-11-21T16:15:43.430Z
Answers.UserDisplayName: NikoNyrh
Answers.Body: <p>Also I would suggest you to have a look at the term &quot;Receptive field&quot; in CNNs(Which actually look at scaled version of what the previous Conv layer looked at while it did a convolving step. Reference is - <a href="https://towardsdatascience.com/intuitively-understanding-convolutions-for-deep-learning-1f6f42faee1" rel="nofollow noreferrer">https://towardsdatascience.com/intuitively-understanding-convolutions-for-deep-learning-1f6f42faee1</a>.</p>
  <p>So Inherently it is looking at what the previous layer looked at for that particular filter all through the input image's overall area</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-04-23T18:15:39.080Z
Answers.LastActivityDate: 2023-04-23T18:15:39.080Z
Answers.UserDisplayName: Udemmy Udemmy
Body: <p>In Convolutional Neural Networks we extract and create abstractified “feature maps” of our given image. My thought was this: We extract things like lines initially. Then from different types of lines we are meant to extract higher order features. However, doesn't this require us to look at multiple feature maps at once? Convolutional layers only apply the filter on one matrix at a time, and the only time, to my knowledge, that these feature maps get looked at together is at the fully connected layer.</p>
  <p>To explain further, if we have an image of a circle we want to recognize, this consists of many lines at different angles. But in a convolutional layer, we have these different filters that will pick up different parts of the circle. Then when we add a second convolutional layer, how can it extract a higher order feature without combining feature maps in some way? Do we combine feature maps in between convolutional layers?</p>
  
Tags: <neural-networks><convolutional-neural-networks><feature-maps>
Title: How do we combine feature maps? CNN
UserDisplayName: Brian Przezdziecki
ThreadText: # How do we combine feature maps? CNN
  
  (Asked by: username_0 on 2022-11-21)
  
  In Convolutional Neural Networks we extract and create abstractified “feature maps” of our given image. My thought was this: We extract things like lines initially. Then from different types of lines we are meant to extract higher order features. However, doesn't this require us to look at multiple feature maps at once? Convolutional layers only apply the filter on one matrix at a time, and the only time, to my knowledge, that these feature maps get looked at together is at the fully connected layer.
  
  To explain further, if we have an image of a circle we want to recognize, this consists of many lines at different angles. But in a convolutional layer, we have these different filters that will pick up different parts of the circle. Then when we add a second convolutional layer, how can it extract a higher order feature without combining feature maps in some way? Do we combine feature maps in between convolutional layers?
  
  **Tags:** neural-networks, convolutional-neural-networks, feature-maps
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2022-11-21. Score: 0 | Accepted answer
  
  I'm not quite sure what you mean by "combining" these maps, but here is a simple example (in Keras):
  model = keras.models.Sequential([
      layers.InputLayer((res, res, 1)),
      layers.Conv2D(3, 7, activation='sigmoid'),
      layers.Conv2D(3, 7, activation='sigmoid'),
      
      layers.GlobalMaxPooling2D(),
      layers.Dense(1, activation='sigmoid')
  ])
  
  Layer (type)                 Output Shape              Param #   
  =================================================================
  conv2d_199 (Conv2D)          (None, 42, 42, 3)         150       
  _________________________________________________________________
  conv2d_200 (Conv2D)          (None, 36, 36, 3)         444       
  _________________________________________________________________
  global_max_pooling2d_92 (Glo (None, 3)                 0         
  _________________________________________________________________
  dense_102 (Dense)            (None, 1)                 4         
  =================================================================
  Total params: 598
  Trainable params: 598
  Non-trainable params: 0
  
  I used sigmoid activations on the convolutional layers as well (with three "kernels"), since their outputs are easy to visualize as RGB images.
  I trained the network on generated 48x48 grayscale images, each showing either an ellipse or a rectangle. Input images are shown on the top right, and they are normalized so that 25% of the pixels are black and 2% are white.
  The intermediate output is quite hard to interpret, but the second output (before global max pooling) correlates well with the input images, and also with the output target class. The green channel seems to correspond to circles and ellipses (detecting curvature?), while red and blue channels react to linear lines. These correlations are also shown in the lower plot.
  
  Then when we add a second convolutional layer, how can it extract a higher order feature without combining feature maps in some way?
  
  Note that while the first Conv2D layer has weights of shape [7, 7, 1, 3] (ignoring the bias), the seconds one has a shape of [7, 7, 3, 3]. Eg. it sees all three channels of the previous layer simultaneously, meaning three "separate" matrices. So "convolutional layers only apply the filter on one matrix at a time" isn't quite true. Instead they apply to one tensor at a time, which can be interpreted as stacked matrices.
  
  Note that the network may learn to detect very different aspects, depending on the initial parameters and specifics of the data. For example in this case the shapes are filled solid, and the network makes the distinction whether there are corners in the image or not. The green channel alone doesn't seem to be good for this classification task.
  
  > **Comments:**
  > **username_0**: Thanks so much this definitely cleared up my confusion!  "So "convolutional layers only apply the filter on one matrix at a time" isn't quite true. Instead they apply to one tensor at a time, which can be interpreted as stacked matrices."  Okay, I see the filter/kernel can also be a tensor. Interesting. So would the order of your feature maps matter? It would seem to me that they would then. In this case where there is 3 channels it wouldn't, but if the kernel was smaller, the order of the feature maps would make some difference right?  
  > **username_0**: Wait actually, no it doesn't. Because the kernels before are learned. Okay sorry if I'm being confusing, don't worry about it I got it now!  
  > **username_1**: Ah, a simpler way to put this is to think of a color image with RGB color channels. The first convolution kernel looks at all of the color channel simultaneously, while seeing just a certain part of the image (depending on the kernel's width and height). And usually the number of channels increases as we add more layers to the network, but the math is still the same. A RGB image can be thought of as three matrices, or a single tensor.  
  
  ---
  
  ### Answer by username_2 on 2023-04-23. Score: 0
  
  Also I would suggest you to have a look at the term "Receptive field" in CNNs(Which actually look at scaled version of what the previous Conv layer looked at while it did a convolving step. Reference is - <https://towardsdatascience.com/intuitively-understanding-convolutions-for-deep-learning-1f6f42faee1>.
  
  So Inherently it is looking at what the previous layer looked at for that particular filter all through the input image's overall area
  
  
  ---
  
  