Answers.Body: <p>I assume that by &quot;tuning parameter&quot; you mean, e.g., the weights of a nueral net. These are parameters that can be learned from data. A hyperparameter, however, can not (or only indirectly) be estimated from your training data. Even though, during hyperparameter optimization you can determine which of these hyperparameters results in a model that is more (or less) appropriate to describe the data -- or you can simply make an educated guess about that parameter. In this sense, the polynomial degree of a regression model is a hyperparameter that needs to be determined before you use the model for training it and, e.g., making predictions or inference with it.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-10-09T14:41:27.023Z
Answers.LastActivityDate: 2023-10-09T14:41:27.023Z
Answers.UserDisplayName: BanDoP
Body: <p>We know that in ML we have tuning parameters and hyperparameters.
  Is model order of a model class (for example, polynomial regression class) a hyperparameter or a tuning parameter?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-10-09T21:56:56.073Z
Comments.Text: What do you mean by "model order of a model class"?
Comments.UserDisplayName: nbro
Tags: <machine-learning><validation>
Title: Is model order of a model class (for example, polynomial regression class) a hyperparameter or a tuning parameter?
UserDisplayName: DSPinfinity
ThreadText: # Is model order of a model class (for example, polynomial regression class) a hyperparameter or a tuning parameter?
  
  (Asked by: username_0 on 2023-10-09)
  
  We know that in ML we have tuning parameters and hyperparameters.
  Is model order of a model class (for example, polynomial regression class) a hyperparameter or a tuning parameter?
  
  **Tags:** machine-learning, validation
  > **Comments:**
  > **username_1**: What do you mean by "model order of a model class"?  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2023-10-09. Score: 1 | Accepted answer
  
  I assume that by "tuning parameter" you mean, e.g., the weights of a nueral net. These are parameters that can be learned from data. A hyperparameter, however, can not (or only indirectly) be estimated from your training data. Even though, during hyperparameter optimization you can determine which of these hyperparameters results in a model that is more (or less) appropriate to describe the data -- or you can simply make an educated guess about that parameter. In this sense, the polynomial degree of a regression model is a hyperparameter that needs to be determined before you use the model for training it and, e.g., making predictions or inference with it.
  
  
  ---
  
  