Body: <p>In Goodfellow et al. book Deep Learning <a href="https://www.deeplearningbook.org/contents/applications.html" rel="nofollow noreferrer">chapter 12.1.4</a> they write</p>
  <blockquote>
  <p>These large models learn some function <span class="math-container">$f(x)$</span>, but do so using many more parameters than are necessary for the task. Their size is necessary only due to the limited number of training examples.</p>
  </blockquote>
  <p>I am not able to understand this. Large models are expressive, but if you train them on few examples they should also overfit.</p>
  <p>So, what do the authors mean by saying large models are necessary precisely because of the limited number of training examples?</p>
  <p>This seems to go against the spirit of using more bias when training data is limited.</p>
  
Tags: <neural-networks><overfitting><bias-variance-tradeoff>
Title: Why are large models necessary when we have a limited number of training examples?
UserDisplayName: Borun Chowdhury
ThreadText: # Why are large models necessary when we have a limited number of training examples?
  
  (Asked by: username_0 on 2020-08-22)
  
  In Goodfellow et al. book Deep Learning [chapter 12.1.4](https://www.deeplearningbook.org/contents/applications.html) they write
  
  > These large models learn some function $f(x)$, but do so using many more parameters than are necessary for the task. Their size is necessary only due to the limited number of training examples.
  
  I am not able to understand this. Large models are expressive, but if you train them on few examples they should also overfit.
  
  So, what do the authors mean by saying large models are necessary precisely because of the limited number of training examples?
  
  This seems to go against the spirit of using more bias when training data is limited.
  
  **Tags:** neural-networks, overfitting, bias-variance-tradeoff
  
  ---
  
  ## 0 answers
  
  