Body: <p>I am going through the book Pattern Recognition by Bishop.</p>
  
  <p>At one point he says</p>
  
  <blockquote>
    <p>For <span class="math-container">$M = 9$</span>, the training set error goes to zero, as we might expect because this polynomial contains 10 degrees of freedom corresponding to the <span class="math-container">$10$</span> coefficients <span class="math-container">$w_0, \dots, w_9$</span>, and so can be tuned exactly to the <span class="math-container">$10$</span> data points in the training set.</p>
  </blockquote>
  
  <p>where <span class="math-container">$M$</span> is the order of the hypothesis function, and <span class="math-container">$w$</span> are the weights of the hypothesis function.</p>
  
  <p>I did not understand how having <span class="math-container">$10$</span> degrees of freedom will tune the model EXACTLY to the <span class="math-container">$10$</span> data points? Does it mean that whenever we have a number of data points in training set equal to the degrees of freedom, the error will be zero?</p>
  
Tags: <machine-learning><comparison><regression>
Title: What is the relationship between degrees of freedom and the size of the training dataset?
UserDisplayName: sage76
ThreadText: # What is the relationship between degrees of freedom and the size of the training dataset?
  
  (Asked by: username_0 on 2019-07-25)
  
  I am going through the book Pattern Recognition by Bishop.
  
  At one point he says
  
  > For $M = 9$, the training set error goes to zero, as we might expect because this polynomial contains 10 degrees of freedom corresponding to the $10$ coefficients $w\_0, \dots, w\_9$, and so can be tuned exactly to the $10$ data points in the training set.
  
  where $M$ is the order of the hypothesis function, and $w$ are the weights of the hypothesis function.
  
  I did not understand how having $10$ degrees of freedom will tune the model EXACTLY to the $10$ data points? Does it mean that whenever we have a number of data points in training set equal to the degrees of freedom, the error will be zero?
  
  **Tags:** machine-learning, comparison, regression
  
  ---
  
  ## 0 answers
  
  