Answers.Body: <p>“Nonparametric” is a controversial term in statistics whose definition lacks a consensus.</p>
  <p>One definition is that the parameter count increases as the data size increases. This is how kernel density estimation works: for each new point, we add a new kernel. Perhaps a neural network could be seen in this light by allowing the number of neurons to increase as data are added.</p>
  <p>One definition is that we allow the data to describe themselves rather than shoehorning them into a particular form that is governed by parameters. A neural network could be seen this way, as the idea is to give a large amount of flexibility to allow the data to extract their own features that involve curvature and variable interactions; we don’t have to specify these like we do in a linear regression or generalized linear model if the model figures them out.</p>
  <p>If the author had referred to a neural network as a particular type of nonlinear regression, then the statement would not be controversial.</p>
  <p>Because of the lack of consensus on what “nonparametric” means, towards the end of 2020, there was a movement/joke on <a href="https://stats.stackexchange.com/">Cross Validated (Statistics Stack Exchange)</a> that our new year’s resolution would be to stop using the term “nonparametric” for exactly that reason.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-08-02T15:30:43.873Z
Answers.LastActivityDate: 2022-08-06T03:06:16.537Z
Answers.LastEditDate: 2022-08-06T03:06:16.537Z
Answers.UserDisplayName: Dave
Body: <p>E. Alpaydin, Introduction to ML, 4-th edition, page 46:</p>
  <blockquote>
  <p>Over time, it has been realized
  in the neural network community that most neural network
  learning algorithms have their basis in statistics—for example,
  the multilayer perceptron is another class of nonparametric
  estimator.</p>
  </blockquote>
  <p>Why is multilayer perceptron a nonparametric model?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-08-01T23:05:29.827Z
Comments.Text: The terms "parametric" and "non-parametric" have been used ambiguously off-late, see discussion - https://stats.stackexchange.com/a/322051/219259. A comment from that answer says " As the wiki-page on nonparametric statistics says, there is no general consensus on what non-parametric means".
Comments.UserDisplayName: Ganesh Tata
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-08-02T15:25:31.243Z
Comments.Text: @GaneshTata Towards the end of 2020, there was a joke in the Cross Validated Meta that our new year’s resolution would be to stop using the term “nonparametric” for exactly that reason.
Comments.UserDisplayName: Dave
Tags: <machine-learning><deep-learning>
Title: Why is multilayer perceptron a nonparametric model?
UserDisplayName: me_asking_Qs
ThreadText: # Why is multilayer perceptron a nonparametric model?
  
  (Asked by: username_0 on 2022-08-01)
  
  E. Alpaydin, Introduction to ML, 4-th edition, page 46:
  
  > Over time, it has been realized
  > in the neural network community that most neural network
  > learning algorithms have their basis in statistics—for example,
  > the multilayer perceptron is another class of nonparametric
  > estimator.
  
  Why is multilayer perceptron a nonparametric model?
  
  **Tags:** machine-learning, deep-learning
  > **Comments:**
  > **username_1**: The terms "parametric" and "non-parametric" have been used ambiguously off-late, see discussion - https://stats.stackexchange.com/a/322051/219259. A comment from that answer says " As the wiki-page on nonparametric statistics says, there is no general consensus on what non-parametric means".  
  > **username_2**: @GaneshTata Towards the end of 2020, there was a joke in the Cross Validated Meta that our new year’s resolution would be to stop using the term “nonparametric” for exactly that reason.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2022-08-02. Score: 7 | Accepted answer
  
  “Nonparametric” is a controversial term in statistics whose definition lacks a consensus.
  
  One definition is that the parameter count increases as the data size increases. This is how kernel density estimation works: for each new point, we add a new kernel. Perhaps a neural network could be seen in this light by allowing the number of neurons to increase as data are added.
  
  One definition is that we allow the data to describe themselves rather than shoehorning them into a particular form that is governed by parameters. A neural network could be seen this way, as the idea is to give a large amount of flexibility to allow the data to extract their own features that involve curvature and variable interactions; we don’t have to specify these like we do in a linear regression or generalized linear model if the model figures them out.
  
  If the author had referred to a neural network as a particular type of nonlinear regression, then the statement would not be controversial.
  
  Because of the lack of consensus on what “nonparametric” means, towards the end of 2020, there was a movement/joke on [Cross Validated (Statistics Stack Exchange)](https://stats.stackexchange.com/) that our new year’s resolution would be to stop using the term “nonparametric” for exactly that reason.
  
  
  ---
  
  