Answers.Body: <p><a href="https://en.wikipedia.org/wiki/Efficiency_(statistics)" rel="noreferrer">Statistical efficiency</a> in this context essentially means that a CNN would require fewer training examples than a fully connected network to learn. Intuitively this seems reasonable: more parameters to learn should mean more samples needed. Of course it is always desirable to minimise the number of training samples needed, so that's a definite advantage of CNNs.</p>
  <p>There is <a href="https://papers.nips.cc/paper/2018/file/03c6b06952c750899bb03d998e631860-Paper.pdf" rel="noreferrer">a paper</a> on the efficiency of CNNs which attempts to make that statement more precise. They examine the case of a convolutional network using a linear activation function.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-04-19T08:26:31.127Z
Answers.LastActivityDate: 2021-04-19T08:26:31.127Z
Answers.UserDisplayName: htl
Body: <p>Consider the following statement(s) from <a href="https://www.deeplearningbook.org/contents/convnets.html" rel="nofollow noreferrer">Deep Learning book (p. 333, chapter 9: Convolutional Networks)</a> by <em>Ian Goodfellow et al</em>.</p>
  <blockquote>
  <p>Convolution is thus dramatically more efficient than dense matrix
  multiplication in terms of the memory requirements and <strong>statistical
  eﬃciency.</strong></p>
  </blockquote>
  <p>Book is saying that statistical efficiency is due to the decrease in the number of parameters due to convolution (using kernel) compared to fully connected feed forward neural networks.</p>
  <p>What is meant by statistical efficiency in this context? And how does decrease in the number of parameters increase statistical efficiency?</p>
  
Tags: <convolutional-neural-networks><terminology><books>
Title: What does "statistical efficiency" mean in this context?
UserDisplayName: hanugm
ThreadText: # What does "statistical efficiency" mean in this context?
  
  (Asked by: username_0 on 2021-04-19)
  
  Consider the following statement(s) from [Deep Learning book (p. 333, chapter 9: Convolutional Networks)](https://www.deeplearningbook.org/contents/convnets.html) by *Ian Goodfellow et al*.
  
  > Convolution is thus dramatically more efficient than dense matrix
  > multiplication in terms of the memory requirements and **statistical
  > eﬃciency.**
  
  Book is saying that statistical efficiency is due to the decrease in the number of parameters due to convolution (using kernel) compared to fully connected feed forward neural networks.
  
  What is meant by statistical efficiency in this context? And how does decrease in the number of parameters increase statistical efficiency?
  
  **Tags:** convolutional-neural-networks, terminology, books
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-04-19. Score: 5 | Accepted answer
  
  [Statistical efficiency](https://en.wikipedia.org/wiki/Efficiency_(statistics)) in this context essentially means that a CNN would require fewer training examples than a fully connected network to learn. Intuitively this seems reasonable: more parameters to learn should mean more samples needed. Of course it is always desirable to minimise the number of training samples needed, so that's a definite advantage of CNNs.
  
  There is [a paper](https://papers.nips.cc/paper/2018/file/03c6b06952c750899bb03d998e631860-Paper.pdf) on the efficiency of CNNs which attempts to make that statement more precise. They examine the case of a convolutional network using a linear activation function.
  
  
  ---
  
  