Answers.Body: <p>Even if it’s impossible to answer this question properly, as <em>non trivial</em> is not well defined (maybe the author will edit this questions later, to specify it better), I take the opportunity to point out this paper which looks interesting to me </p>
  
  <p><a href="https://arxiv.org/abs/1804.02171" rel="noreferrer">Smallest Neural Network to Learn the Ising Criticality</a></p>
  
  <p>Assuming you have a general idea of the <a href="https://en.wikipedia.org/wiki/Ising_model" rel="noreferrer">Ising Model</a> I think the problem of identifying the critical temperature from a data driven perspective can be considered as <em>non trivial</em> and the paper shows how the authors have improved the performance related to solve this task with NN passing from 100 Hidden Neurons, as performed in this paper <a href="https://www.nature.com/articles/nphys4035" rel="noreferrer">Machine learning phases of matter</a> from 2017, to only 2 Hidden Neurons </p>
  
  <p>Just my cents: </p>
  
  <ul>
  <li>reducing the neurons, while keeping good performance, should help in terms of <strong>neural processing interpretability</strong> which is notoriously obscure and its complexity grows (exponentially) with the number of neurons</li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-01T20:12:51.493Z
Answers.Text: I just wanted examples that the community itself found interesting. I think it is actually non trivial to define non triviality. Right?
Answers.UserDisplayName: Guillermo Mosse
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-02T03:24:09.553Z
Answers.Text: @GuillermoMosse No, it's quite trivial to define non-trivial. The definition is more or less arbitrary depending on what best fits the situation. I'm sure a very simple NN could learn to tell whether or not a given number is a non-trivial square root...
Answers.UserDisplayName: forest
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-01T15:35:52.330Z
Answers.LastActivityDate: 2019-03-01T17:43:18.420Z
Answers.LastEditDate: 2019-03-01T17:43:18.420Z
Answers.UserDisplayName: Nicola Bernini
Body: <p>I'm interested in knowing whether there exist any neural network, that solves (with  >=80% accuracy) any nontrivial problem, that uses very few nodes (where 20 nodes is not a hard limit). I want to develop an intuition on sizes of neural networks.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-03-01T15:06:51.757Z
Comments.Text: Universal approximation theorem: a neural network with one hidden layer can approximate any "reasonable" function given a sufficient number of nodes in the hidden layer.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-03-01T15:08:03.310Z
Comments.Text: I think it will not be easy to answer your question. What do you mean by "non-trivial problem"?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-03-01T20:11:57.443Z
Comments.Text: I want few nodes even in hidden layers.
Comments.UserDisplayName: Guillermo Mosse
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-03-01T22:54:19.713Z
Comments.Text: @nbro my sense of "non-trivial" in this context is intractable or unsolved.
Comments.UserDisplayName: DukeZhou
Tags: <neural-networks>
Title: Are there neural networks with very few nodes that decently solve non-trivial problems?
UserDisplayName: Guillermo Mosse
ThreadText: # Are there neural networks with very few nodes that decently solve non-trivial problems?
  
  (Asked by: username_0 on 2019-03-01)
  
  I'm interested in knowing whether there exist any neural network, that solves (with >=80% accuracy) any nontrivial problem, that uses very few nodes (where 20 nodes is not a hard limit). I want to develop an intuition on sizes of neural networks.
  
  **Tags:** neural-networks
  
  > **Comments:**
  > **username_1**: Universal approximation theorem: a neural network with one hidden layer can approximate any "reasonable" function given a sufficient number of nodes in the hidden layer.  
  > **username_1**: I think it will not be easy to answer your question. What do you mean by "non-trivial problem"?  
  > **username_0**: I want few nodes even in hidden layers.  
  > **username_2**: @username_1 my sense of "non-trivial" in this context is intractable or unsolved.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2019-03-01. Score: 13 | Accepted answer
  
  Even if it’s impossible to answer this question properly, as *non trivial* is not well defined (maybe the author will edit this questions later, to specify it better), I take the opportunity to point out this paper which looks interesting to me
  
  [Smallest Neural Network to Learn the Ising Criticality](https://arxiv.org/abs/1804.02171)
  
  Assuming you have a general idea of the [Ising Model](https://en.wikipedia.org/wiki/Ising_model) I think the problem of identifying the critical temperature from a data driven perspective can be considered as *non trivial* and the paper shows how the authors have improved the performance related to solve this task with NN passing from 100 Hidden Neurons, as performed in this paper [Machine learning phases of matter](https://www.nature.com/articles/nphys4035) from 2017, to only 2 Hidden Neurons
  
  Just my cents:
  
  * reducing the neurons, while keeping good performance, should help in terms of **neural processing interpretability** which is notoriously obscure and its complexity grows (exponentially) with the number of neurons
  
  > **Comments:**
  > **username_0**: I just wanted examples that the community itself found interesting. I think it is actually non trivial to define non triviality. Right?  
  > **username_4**: @username_0 No, it's quite trivial to define non-trivial. The definition is more or less arbitrary depending on what best fits the situation. I'm sure a very simple NN could learn to tell whether or not a given number is a non-trivial square root...  
  
  ---
  
  