Answers.Body: <p><strong>Context of the question</strong></p>
  <p>This is a <a href="https://livebook.manning.com/book/deep-learning-with-python/chapter-7/v-6/1" rel="nofollow noreferrer">link</a> to the text cited in the question.</p>
  <p>It refers to the usage of SeparableConv2D (tf, keras name). A related question on StackOverflow is &quot;<a href="https://stackoverflow.com/questions/54705367/what-is-the-difference-between-separableconv2d-and-conv2d-layers">What is the difference between SeparableConv2D and Conv2D layers</a>&quot;. This answer points to this excellent article by Chi-Feng Wang:</p>
  <p><a href="https://towardsdatascience.com/a-basic-introduction-to-separable-convolutions-b99ec3102728" rel="nofollow noreferrer">A Basic Introduction to Separable Convolutions</a></p>
  <p><strong>Answer to the question</strong></p>
  <p>In image processing, a separable convolution converts a NxM convolution to two convolutions with kernels Nx1 and 1xM. Using this idea, in NN a SeparableConv2D converts a WxHxD convolution (width x height x depth, where depth means number of incoming features ) to two convolutions with kernels WxHx1 and 1x1xD.</p>
  <p>Note the first kernel doesn't handles information across features, thus, it is &quot;learning of spatial features&quot;. The 1x1xD kernel doesn't handles different points, it is &quot;learning of channel-wise features&quot;.</p>
  <p>About the phrase &quot;spatial locations in the input are highly correlated&quot;, my understanding of what the author means is: Assume we have a channel (feature) image that each pixel measures the &quot;distance to the background&quot;. When we pass from one pixel to a neighbors one, it is expected some continuity in the value (except for edge pixels): correlation. Instead, if we have a channel that measures &quot;brightness&quot; and another one that measures &quot;distance to background&quot; the two values for one specific pixel has little correlation.</p>
  <p>Finally, about title question &quot;When should we use separable convolution?&quot; : if the final output must depend of some features of one pixel and some other features of neighbors pixels in a very unpredictable way, a complete WxHxD convolution must be used. However if, as is more usual, you can handle first spatial dependencies (neighborhood) to extract pixel features and next handle pixel-by-pixel these features to get the output, better use a WxHx1 followed by 1x1xD, saving lots of network parameters, thus, saving training time.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-08-19T18:11:27.153Z
Answers.Text: "a complete WxHxD convolution must be used" Why? A DSC has the same receptive field as a normal convolution so should, in theory, always be able to replace it.
Answers.UserDisplayName: Gaslight Deceive Subvert
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-17T14:46:04.717Z
Answers.LastActivityDate: 2020-10-18T15:03:32.537Z
Answers.LastEditDate: 2020-10-18T15:03:32.537Z
Answers.UserDisplayName: pasaba por aqui
Body: <p>I was reading the <em>&quot;Deep Learning with Python&quot;</em> by François Chollet. He mentioned separable convolution as following</p>
  <blockquote>
  <p>This is equivalent to separating the learning of spatial features and
  the learning of channel-wise features, which makes a lot of sense if
  you assume that spatial locations in the input are highly correlated,
  but different channels are fairly independent.</p>
  </blockquote>
  <p>But I could not understand what he meant by saying &quot;correlated spatial locations&quot;. Can some explain what he means or the purpose of separable convolutions? (except performance-related part).</p>
  <p>Edit: Separable convolution means that first depthwise convolution is applied then pointwise convolution is applied.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-10-16T22:56:27.330Z
Comments.Text: To explain this part "_spatial locations in the input are highly correlated_", i.e. that pixels in some neighbourhood of a certain channel are likely to be correlated, imagine that in some neighbourhood of the channel you have the face of a person, then it should be clear that all pixels that belong to this face are correlated, in the sense that they will have very similar intensity value (for the respective channel). I can't answer why channels would be independent without more context. Maybe they would be independent because they are not encoding the RGB values but something else.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-10-17T13:47:15.200Z
Comments.Text: @nbro It does not have to be RGB, lets say we apply separable convolution as second layer. In this context channel number comes from number of filter we applied as you probably know? The example you gave did not clarify the problem.
Comments.UserDisplayName: Enes
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-10-17T15:25:45.083Z
Comments.Text: Why doesn't my example clarify what the authors mean by "spatial correlation"?
Comments.UserDisplayName: nbro
Tags: <convolutional-neural-networks><convolution>
Title: When should we use separable convolution?
UserDisplayName: Enes
ThreadText: # When should we use separable convolution?
  
  (Asked by: username_0 on 2020-10-14)
  
  I was reading the *"Deep Learning with Python"* by François Chollet. He mentioned separable convolution as following
  
  > This is equivalent to separating the learning of spatial features and
  > the learning of channel-wise features, which makes a lot of sense if
  > you assume that spatial locations in the input are highly correlated,
  > but different channels are fairly independent.
  
  But I could not understand what he meant by saying "correlated spatial locations". Can some explain what he means or the purpose of separable convolutions? (except performance-related part).
  
  Edit: Separable convolution means that first depthwise convolution is applied then pointwise convolution is applied.
  
  **Tags:** convolutional-neural-networks, convolution
  
  > **Comments:**
  > **username_1**: To explain this part "_spatial locations in the input are highly correlated_", i.e. that pixels in some neighbourhood of a certain channel are likely to be correlated, imagine that in some neighbourhood of the channel you have the face of a person, then it should be clear that all pixels that belong to this face are correlated, in the sense that they will have very similar intensity value (for the respective channel). I can't answer why channels would be independent without more context. Maybe they would be independent because they are not encoding the RGB values but something else.  
  > **username_0**: @username_1 It does not have to be RGB, lets say we apply separable convolution as second layer. In this context channel number comes from number of filter we applied as you probably know? The example you gave did not clarify the problem.  
  > **username_1**: Why doesn't my example clarify what the authors mean by "spatial correlation"?  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-10-17. Score: 4 | Accepted answer
  
  **Context of the question**
  
  This is a [link](https://livebook.manning.com/book/deep-learning-with-python/chapter-7/v-6/1) to the text cited in the question.
  
  It refers to the usage of SeparableConv2D (tf, keras name). A related question on StackOverflow is "[What is the difference between SeparableConv2D and Conv2D layers](https://stackoverflow.com/questions/54705367/what-is-the-difference-between-separableconv2d-and-conv2d-layers)". This answer points to this excellent article by Chi-Feng Wang:
  
  [A Basic Introduction to Separable Convolutions](https://towardsdatascience.com/a-basic-introduction-to-separable-convolutions-b99ec3102728)
  
  **Answer to the question**
  
  In image processing, a separable convolution converts a NxM convolution to two convolutions with kernels Nx1 and 1xM. Using this idea, in NN a SeparableConv2D converts a WxHxD convolution (width x height x depth, where depth means number of incoming features ) to two convolutions with kernels WxHx1 and 1x1xD.
  
  Note the first kernel doesn't handles information across features, thus, it is "learning of spatial features". The 1x1xD kernel doesn't handles different points, it is "learning of channel-wise features".
  
  About the phrase "spatial locations in the input are highly correlated", my understanding of what the author means is: Assume we have a channel (feature) image that each pixel measures the "distance to the background". When we pass from one pixel to a neighbors one, it is expected some continuity in the value (except for edge pixels): correlation. Instead, if we have a channel that measures "brightness" and another one that measures "distance to background" the two values for one specific pixel has little correlation.
  
  Finally, about title question "When should we use separable convolution?" : if the final output must depend of some features of one pixel and some other features of neighbors pixels in a very unpredictable way, a complete WxHxD convolution must be used. However if, as is more usual, you can handle first spatial dependencies (neighborhood) to extract pixel features and next handle pixel-by-pixel these features to get the output, better use a WxHx1 followed by 1x1xD, saving lots of network parameters, thus, saving training time.
  
  > **Comments:**
  > **username_3**: "a complete WxHxD convolution must be used" Why? A DSC has the same receptive field as a normal convolution so should, in theory, always be able to replace it.  
  
  ---
  
  