Answers.Body: <p>I've found out rather a good explanation on <a href="https://qr.ae/pGU8Y2" rel="nofollow noreferrer">Quora</a>.</p>
  <p><a href="https://i.sstatic.net/m5rea.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/m5rea.png" alt="enter image description here" /></a></p>
  <p><strong>Max pooling</strong> extracts the most salient features - edges, cusps, whatever.</p>
  <p><strong>Average pooling</strong> operates smoothly - collects features, that are relevant to any part of the image.</p>
  <p><strong>Max pooling</strong> throws some information, It can be thought as some sense of &quot;forgetting&quot;, whereas <strong>Average pooling</strong> depends on the whole input, despite the output representation is compressed.</p>
  <p>There are cases, where each of these operations may not be good for feature extracting (from <a href="https://www.researchgate.net/figure/Toy-example-illustrating-the-drawbacks-of-max-pooling-and-average-pooling_fig2_300020038" rel="nofollow noreferrer">here</a>):
  <a href="https://i.sstatic.net/JB0Hs.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/JB0Hs.png" alt="enter image description here" /></a></p>
  <p>In th first case <strong>Max Pooling</strong> will produce simply a white background, and in the second after <strong>Average pooling</strong> one will get a pale grey strip (although I think, it should be lighter, than depicted).</p>
  <p>Probably It would make sense, to mix these two approaches, and pool half of the filters with <strong>Max Pooling</strong>, and another half with <strong>Average Pooling</strong>, although, I am not aware of the use of this approach in the literature.</p>
  <p>The most flexible and expressive approach is the stridden convolution and the Average pooling, for example, is a particular case of it, but it introduces a certain (although not big) additional cost for storage of new parameters.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-18T20:26:31.587Z
Answers.LastActivityDate: 2021-10-18T20:26:31.587Z
Answers.UserDisplayName: spiridon_the_sun_rotator
Body: <p>Consider the following excerpt taken from the chapter named <strong><a href="https://pytorch.org/assets/deep-learning/Deep-Learning-with-PyTorch.pdf#page=233" rel="nofollow noreferrer">Using convolutions to generalize</a></strong> from the textbook titled <strong>Deep Learning with PyTorch</strong> by <em>Eli Stevens et al</em>.</p>
  <blockquote>
  <p>Downsampling could in principle occur in different ways. Scaling an
  image by half is the equivalent of taking four neighboring pixels as
  input and producing one pixel as output. How we compute the value of
  the output based on the values of the input is up to us. We could</p>
  <ul>
  <li>Average the four pixels. This average pooling was a common approach early on <strong>but has fallen out of favor somewhat.</strong></li>
  <li>Take the maximum of the four pixels. This approach, called max pooling, <strong>is currently the most commonly used approach</strong>, but it has
  a downside of discarding the other three-quarters of the data.</li>
  <li>Perform a strided convolution, where only every <span class="math-container">$N$</span>-th pixel is calculated. A <span class="math-container">$3 \times 4$</span> convolution with stride 2 still incorporates input
  from all pixels from the previous layer. The literature shows promise
  for this approach, but it has not yet supplanted max pooling.</li>
  </ul>
  </blockquote>
  <p>The paragraph is mentioning that the research community is biased towards max-pooling than average pooling. Is there any rational basis for such bias?</p>
  
Tags: <deep-learning><max-pooling><pooling><average-pooling>
Title: Is there any reason behind bias towards max pooling over avg pooling?
UserDisplayName: hanugm
ThreadText: # Is there any reason behind bias towards max pooling over avg pooling?
  
  (Asked by: username_0 on 2021-10-15)
  
  Consider the following excerpt taken from the chapter named **[Using convolutions to generalize](https://pytorch.org/assets/deep-learning/Deep-Learning-with-PyTorch.pdf#page=233)** from the textbook titled **Deep Learning with PyTorch** by *Eli Stevens et al*.
  
  > Downsampling could in principle occur in different ways. Scaling an
  > image by half is the equivalent of taking four neighboring pixels as
  > input and producing one pixel as output. How we compute the value of
  > the output based on the values of the input is up to us. We could
  >
  > * Average the four pixels. This average pooling was a common approach early on **but has fallen out of favor somewhat.**
  > * Take the maximum of the four pixels. This approach, called max pooling, **is currently the most commonly used approach**, but it has
  >   a downside of discarding the other three-quarters of the data.
  > * Perform a strided convolution, where only every $N$-th pixel is calculated. A $3 \times 4$ convolution with stride 2 still incorporates input
  >   from all pixels from the previous layer. The literature shows promise
  >   for this approach, but it has not yet supplanted max pooling.
  
  The paragraph is mentioning that the research community is biased towards max-pooling than average pooling. Is there any rational basis for such bias?
  
  **Tags:** deep-learning, max-pooling, pooling, average-pooling
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-10-18. Score: 2
  
  I've found out rather a good explanation on [Quora](https://qr.ae/pGU8Y2).
  
  [![enter image description here](https://i.sstatic.net/m5rea.png)](https://i.sstatic.net/m5rea.png)
  
  **Max pooling** extracts the most salient features - edges, cusps, whatever.
  
  **Average pooling** operates smoothly - collects features, that are relevant to any part of the image.
  
  **Max pooling** throws some information, It can be thought as some sense of "forgetting", whereas **Average pooling** depends on the whole input, despite the output representation is compressed.
  
  There are cases, where each of these operations may not be good for feature extracting (from [here](https://www.researchgate.net/figure/Toy-example-illustrating-the-drawbacks-of-max-pooling-and-average-pooling_fig2_300020038)):
  [![enter image description here](https://i.sstatic.net/JB0Hs.png)](https://i.sstatic.net/JB0Hs.png)
  
  In th first case **Max Pooling** will produce simply a white background, and in the second after **Average pooling** one will get a pale grey strip (although I think, it should be lighter, than depicted).
  
  Probably It would make sense, to mix these two approaches, and pool half of the filters with **Max Pooling**, and another half with **Average Pooling**, although, I am not aware of the use of this approach in the literature.
  
  The most flexible and expressive approach is the stridden convolution and the Average pooling, for example, is a particular case of it, but it introduces a certain (although not big) additional cost for storage of new parameters.
  
  
  ---
  
  