Answers.Body: <p>With exactly one hidden neuron, I believe the answer is no. The hidden neuron has one input and one output, and thus all it does is scale the final output by its activation function. Thus, it does not accomplish anything.</p>
  <p>However, with a very slight tweak, the answer is yes. If we instead consider a network with exactly 2 hidden neurons, this becomes a very famous problem.</p>
  <p>The XOR problem cannot be solved without a hidden layer, but is solvable with one hidden layer consisting of 2 neurons.</p>
  <p>Famously, this limitation of single layer perceptrons led Marvin Minsky to be very critical of research on Neural Networks in the 1970s, and was a contributor to the first AI winter.</p>
  <p>Here is an example of an XOR, which could not be computed with no hidden layer:
  <a href="https://i.sstatic.net/8A5bh.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/8A5bh.png" alt="enter image description here" /></a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-23T16:32:00.373Z
Answers.LastActivityDate: 2023-03-23T16:32:00.373Z
Answers.UserDisplayName: chessprogrammer
Answers.Body: <p>With one hidden node, there is one single input to the output neuron.</p>
  <p>Therefore, the two neurons are equivalent to a single neuron with activation function the composition of the two activation functions.</p>
  <p>This collapses the two layers into one, so no problem can be solved with only one single hidden neuron that is not solved with no hidden layer.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-23T16:21:51.923Z
Answers.Text: You should clarify that the answer to the OP's question is basically "no" due to their constraint.
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-23T16:12:51.607Z
Answers.LastActivityDate: 2023-03-23T16:52:48.053Z
Answers.LastEditDate: 2023-03-23T16:52:48.053Z
Answers.UserDisplayName: Jaume Oliver Lafont
Body: <p>Basically, I'm wondering if there are any small and simple problems that are:</p>
  <ul>
  <li>complex enough to be <strong>unsolvable</strong> with a standard neural network without any hidden layer (ie. input -&gt; output)</li>
  <li>simple enough to be <strong>solvable</strong> with a standard neural network consisting on exactly one hidden node in exactly one hidden layer (ie. input -&gt; one hidden node -&gt; output)</li>
  </ul>
  <p>Can such problems even exist at all? If not, why?</p>
  
Tags: <neural-networks><hidden-layers><logistic-regression>
Title: Are there any toy classification problems that can't be solved with logistic regression, but can be solved with a NN with exactly one hidden node?
UserDisplayName: J Doug
ThreadText: # Are there any toy classification problems that can't be solved with logistic regression, but can be solved with a NN with exactly one hidden node?
  
  (Asked by: username_0 on 2023-03-22)
  
  Basically, I'm wondering if there are any small and simple problems that are:
  
  * complex enough to be **unsolvable** with a standard neural network without any hidden layer (ie. input -> output)
  * simple enough to be **solvable** with a standard neural network consisting on exactly one hidden node in exactly one hidden layer (ie. input -> one hidden node -> output)
  
  Can such problems even exist at all? If not, why?
  
  **Tags:** neural-networks, hidden-layers, logistic-regression
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2023-03-23. Score: 0 | Accepted answer
  
  With exactly one hidden neuron, I believe the answer is no. The hidden neuron has one input and one output, and thus all it does is scale the final output by its activation function. Thus, it does not accomplish anything.
  
  However, with a very slight tweak, the answer is yes. If we instead consider a network with exactly 2 hidden neurons, this becomes a very famous problem.
  
  The XOR problem cannot be solved without a hidden layer, but is solvable with one hidden layer consisting of 2 neurons.
  
  Famously, this limitation of single layer perceptrons led Marvin Minsky to be very critical of research on Neural Networks in the 1970s, and was a contributor to the first AI winter.
  
  Here is an example of an XOR, which could not be computed with no hidden layer:
  [![enter image description here](https://i.sstatic.net/8A5bh.png)](https://i.sstatic.net/8A5bh.png)
  
  
  ---
  
  ### Answer by username_2 on 2023-03-23. Score: 0
  
  With one hidden node, there is one single input to the output neuron.
  
  Therefore, the two neurons are equivalent to a single neuron with activation function the composition of the two activation functions.
  
  This collapses the two layers into one, so no problem can be solved with only one single hidden neuron that is not solved with no hidden layer.
  
  > **Comments:**
  > **username_3**: You should clarify that the answer to the OP's question is basically "no" due to their constraint.  
  
  ---
  
  