Answers.Body: <p>They are not close, not anymore!</p>
  
  <p>[Artificial] Neural Nets <strong><em>vaguely inspired</em></strong> by the connections we previously observed between the neurons of a brain. Initially, there probably was an intention to develop ANN to approximate biological brains. However, the modern working ANNs that we see their applications in various tasks are not designed to provide us a functional model of an animal brain. As far as I know, there is no study claiming they have found something new in a biological brain by looking into the connections and weight distributions of let's say a CNN or RNN model.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-04-09T02:41:03.660Z
Answers.LastActivityDate: 2018-04-09T02:41:03.660Z
Answers.UserDisplayName: Borhan Kazimipour
Answers.Body: <p>The common statement that Artificial Neural Networks are inspired by the neural structure of brains is only partially true.</p>
  
  <p>It is true that Norbert Wiener, Claude Shannon, John von Neuman, and others began the path toward practical AI by developing what they then called the electronic brain. It is also true</p>
  
  <ul>
  <li>Artificial networks have functions called activations,</li>
  <li>Are wired in many-to-many relationships like biological neurons, and</li>
  <li>Are designed to learn an optimal behavior,</li>
  </ul>
  
  <p>but that is the extent of the similarity. Cells in artificial networks such as MLPs (multilayer perceptrons) or RNN (Recurrent neural networks) are not like cells in brain networks.</p>
  
  <p>The perceptron, the first software stab at arrays of things that activate, was not an array of neurons. It was the application of basic feedback involving gradients, which had been in common use in engineering ever since James Watt's centrifugal governor was mathematically modeled by Gauss. Successive approximation, a principle that had been in use for centuries, was employed to incrementally update an attenuation matrix. The matrix was multiplied by the vector feeding an array of identical activation functions to produce output. That's it.</p>
  
  <p>The projection in a second dimension to a multi-layer topology was made possible by the realization that the Jacobian could be used to produce a corrective signal that, when distributed as negative feedback to the layers appropriately, could tune the attenuation matrix of a sequence of perceptrons and the network as a whole would converge upon satisfactory behavior. In the sequence of perceptrons, each element is called a layer.  The feedback mechanism is now called back propagation.</p>
  
  <p>The mathematics used to correct the network is called gradient descent because it is like a dehydrated blind man using the gradient of the terrain to find water, and the issues of doing that are similar too.  He might find a local minima (low point) before he finds fresh water and converge on death rather than hydration.</p>
  
  <p>The newer topologies are the additions of already existing convolution work used in digital image restoration, mail sorting, and graphics applications to create the CNN family of topologies and the ingenious use of what is like a chemical equilibrium from first year chemistry to combine optimization criteria creating the GAN family of topologies.</p>
  
  <p>Deep is simply a synonym for numerous in most AI contexts.  It sometimes infers complexity in the higher level topology (above the vector-matrix products, the activations, and the convolutions).</p>
  
  <p>Active research is ongoing by those who are aware how different these deep networks are from what neural scientists have discovered decades ago in mammalian brain tissue.  And there are more differentiators being discovered today as learning circuitry and neuro-chemistry in the brain is investigated from the genomic perspective.</p>
  
  <ul>
  <li>Neural plasticity ... change in circuit topology due to dendrite and axiom growth, death, redirection, and other morphing</li>
  <li>Topological complexity ... large number of axioms crisscross without interacting and are deliberately shielded from cross-talk (independent) most likely because it would be disadvantageous to let them connect <sup>[note 1]</sup></li>
  <li>Chemical signaling ... mammalian brains have dozens of neuro-transmitter and neuro-regulation compounds that have regional effects on circuitry <sup>[note 2]</sup></li>
  <li>Organelles ... living cells have many substructures and it is known that several types have complex relationships with signal transmission in neurons</li>
  <li>Entirely different form of activation ... activations in common artificial neural nets are simply functions with ordinal scalars for both range and domain ... mammalian neurons operate as a function of both amplitude and relative temporal proximity of incoming signals <sup>[note 3]</sup></li>
  </ul>
  
  <p>[1] Topology is ironically both a subset of architecture (in the fields of building design, network provisioning, WWW analysis, and semantic networks), yet at the same time topology is, much more than architecture, at the radical center of both AI mathematics and effective actualization in control systems</p>
  
  <p>[2] The role of chemistry may be essential to learning social and reproductive behavior that interrelates with DNA information propagation, linking in complex ways learning at the level of an ecosystem and the brain.  Furthermore, long term and short term learning divides the brain's learning into two distinct capabilities too.</p>
  
  <p>[3] The impact of the timing of incoming signals on biological neuron activation is understood to some degree, but it may impact much more than neuron output.  It may impact placticity and chemistry too, and the organelles may play a role in that.</p>
  
  <p><strong>Summary</strong></p>
  
  <p>What machine learning libraries do is as much simulating the human brain as Barbie and Ken dolls simulate a real couple.</p>
  
  <p>Nonetheless, remarkable things are arising in the field of deep learning, and it would not surprise me if autonomous vehicles become fully autonomous in our lifetimes.  I would not recommend to any students to become a developer either.  Computers will probably code much better than humans and orders of magnitude faster, and possibly soon.  Some tasks are not of the kind that biology has evolved to do and computers can exceed human capabilities after only a few decades of research, eventually exceeding human performance by several orders of magnitude.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-19T18:05:16.830Z
Answers.Text: There is a drastic difference between driving a car and programming a computer. Driving a car is a well-defined task, so there may be a way to express it as a function approximation problem, which is why neural networks may chime in. The function of using the car controls based on the input does exist and may be approximated. While programming a computer is not really a task, it is a sequence of decisions that one does while thinking about a situation. It is an art to turn troubles into problems.
Answers.UserDisplayName: Evgeniy
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-23T17:05:42.730Z
Answers.LastActivityDate: 2019-01-11T15:40:00.573Z
Answers.LastEditDate: 2019-01-11T15:40:00.573Z
Answers.UserDisplayName: Douglas Daseeco
Answers.Body: <p>We all know that artificial neural networks (ANNs) are <strong>inspired by</strong> but most of them are only <strong>loosely based on</strong> the biological neural networks (BNNs).</p>
  <p>We can analyze the differences and similarities between ANNs and BNNs in terms of the following components.</p>
  <h3>Neurons</h3>
  <p>The following diagram illustrates a biological neuron (screenshot of an image from <a href="https://opentextbc.ca/introductiontopsychology/chapter/3-1-the-neuron-is-the-building-block-of-the-nervous-system/" rel="nofollow noreferrer">this book</a>).</p>
  <p><a href="https://i.sstatic.net/VSOKk.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/VSOKk.jpg" alt="enter image description here" /></a></p>
  <p>The following one illustrates a typical artificial neuron of an ANN (screenshot of figure 1.14 of <a href="http://page.mi.fu-berlin.de/rojas/neural/chapter/K1.pdf" rel="nofollow noreferrer">this book</a>).</p>
  <p><a href="https://i.sstatic.net/ASLAg.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/ASLAg.png" alt="enter image description here" /></a></p>
  <h3>Initialization</h3>
  <p>In the case of an ANN, the initial state and weights are assigned randomly. While for BNNs, the strengths of connections between neurons and the structure of connections don't start as random. The initial state is genetically derived and is the byproduct of evolution.</p>
  <h3>Learning</h3>
  <p>In BNN, learning comes from the interconnections between myriad neurons in the brain. These interconnections change configuration when the brain experiences new stimuli. The changes result in new connections, strengthening of existing connections, and removal of old and unused ones.</p>
  <p>ANNs are trained from scratch usually using a fixed topology (remember topology changes in case of BNNs), although the topology of ANN can also change (for example, take a look at NEAT or some continual learning techniques), which depends on the problem being solved. The weights of an ANN are randomly initialized and adjusted via an optimization algorithm.</p>
  <h3>Number of neurons</h3>
  <p>Another difference (although this difference is always smaller) is in the number of neurons in the network. A typical ANN consists of hundreds, thousands, millions, and, in some exceptional (e.g. GPT-3), billions of neurons. The BNN of the human brain consists of billions. This number varies from animal to animal.</p>
  <h2>Further reading</h2>
  <p>You can find more information <a href="https://news.sophos.com/en-us/2017/09/21/man-vs-machine-comparing-artificial-and-biological-neural-networks/" rel="nofollow noreferrer">here</a> or <a href="https://ai.stackexchange.com/q/258/2444">here</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-20T00:49:13.523Z
Answers.Text: Given that this answer (which is now a wiki) was accepted and it contains some potentially inaccurate claims about biological neural networks, reliable references (e.g. research papers published in Nature or books) are needed to support these claims, in order to avoid more misconceptions and misinformation. Moreover, this answer is incomplete. So, for any reader that is familiar with ANNs, partially familiar with BNNs, and is interested in this, you can try to improve this answer by following my suggestions: proposed an edit, in case you don't have the privilege, or edit the answer directly.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-04-09T08:54:34.713Z
Answers.LastActivityDate: 2021-01-20T11:14:34.737Z
Answers.LastEditDate: 2021-01-20T11:14:34.737Z
Answers.UserDisplayName: Ugnes
Body: <p>I've heard multiple times that "Neural Networks are the best approximation we have to model the human brain", and I think it is commonly known that Neural Networks are modelled after our brain.</p>
  
  <p>I strongly suspect that this model has been simplified, but how much? </p>
  
  <p>How much does, say, the vanilla NN differ from what we know about the human brain? Do we even know?</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-04-08T21:42:40.980Z
Comments.Text: This question was also asked here [What is the difference between biological and artificial neural networks?](https://psychology.stackexchange.com/questions/7880/what-is-the-difference-between-biological-and-artificial-neural-networks).
Comments.UserDisplayName: Andrew Butler
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-20T01:19:58.133Z
Comments.Text: [This chapter](http://page.mi.fu-berlin.de/rojas/neural/chapter/K1.pdf) may be a good reading.
Comments.UserDisplayName: nbro
Tags: <neural-networks><comparison><brain>
Title: How are Artificial Neural Networks and the Biological Neural Networks similar and different?
UserDisplayName: Andreas Storvik Strauman
ThreadText: # How are Artificial Neural Networks and the Biological Neural Networks similar and different?
  
  (Asked by: username_0 on 2018-04-08)
  
  I've heard multiple times that "Neural Networks are the best approximation we have to model the human brain", and I think it is commonly known that Neural Networks are modelled after our brain.
  
  I strongly suspect that this model has been simplified, but how much?
  
  How much does, say, the vanilla NN differ from what we know about the human brain? Do we even know?
  
  **Tags:** neural-networks, comparison, brain
  
  > **Comments:**
  > **username_1**: This question was also asked here [What is the difference between biological and artificial neural networks?](https://psychology.stackexchange.com/questions/7880/what-is-the-difference-between-biological-and-artificial-neural-networks).  
  > **username_2**: [This chapter](http://page.mi.fu-berlin.de/rojas/neural/chapter/K1.pdf) may be a good reading.  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_6 on 2018-04-09. Score: 12 | Accepted answer
  
  We all know that artificial neural networks (ANNs) are **inspired by** but most of them are only **loosely based on** the biological neural networks (BNNs).
  
  We can analyze the differences and similarities between ANNs and BNNs in terms of the following components.
  
  ### Neurons
  
  The following diagram illustrates a biological neuron (screenshot of an image from [this book](https://opentextbc.ca/introductiontopsychology/chapter/3-1-the-neuron-is-the-building-block-of-the-nervous-system/)).
  
  [![enter image description here](https://i.sstatic.net/VSOKk.jpg)](https://i.sstatic.net/VSOKk.jpg)
  
  The following one illustrates a typical artificial neuron of an ANN (screenshot of figure 1.14 of [this book](http://page.mi.fu-berlin.de/rojas/neural/chapter/K1.pdf)).
  
  [![enter image description here](https://i.sstatic.net/ASLAg.png)](https://i.sstatic.net/ASLAg.png)
  
  ### Initialization
  
  In the case of an ANN, the initial state and weights are assigned randomly. While for BNNs, the strengths of connections between neurons and the structure of connections don't start as random. The initial state is genetically derived and is the byproduct of evolution.
  
  ### Learning
  
  In BNN, learning comes from the interconnections between myriad neurons in the brain. These interconnections change configuration when the brain experiences new stimuli. The changes result in new connections, strengthening of existing connections, and removal of old and unused ones.
  
  ANNs are trained from scratch usually using a fixed topology (remember topology changes in case of BNNs), although the topology of ANN can also change (for example, take a look at NEAT or some continual learning techniques), which depends on the problem being solved. The weights of an ANN are randomly initialized and adjusted via an optimization algorithm.
  
  ### Number of neurons
  
  Another difference (although this difference is always smaller) is in the number of neurons in the network. A typical ANN consists of hundreds, thousands, millions, and, in some exceptional (e.g. GPT-3), billions of neurons. The BNN of the human brain consists of billions. This number varies from animal to animal.
  
  ## Further reading
  
  You can find more information [here](https://news.sophos.com/en-us/2017/09/21/man-vs-machine-comparing-artificial-and-biological-neural-networks/) or [here](https://ai.stackexchange.com/q/258/2444).
  
  > **Comments:**
  > **username_2**: Given that this answer (which is now a wiki) was accepted and it contains some potentially inaccurate claims about biological neural networks, reliable references (e.g. research papers published in Nature or books) are needed to support these claims, in order to avoid more misconceptions and misinformation. Moreover, this answer is incomplete. So, for any reader that is familiar with ANNs, partially familiar with BNNs, and is interested in this, you can try to improve this answer by following my suggestions: proposed an edit, in case you don't have the privilege, or edit the answer directly.  
  
  ---
  
  ### Answer by username_3 on 2018-04-09. Score: 5
  
  They are not close, not anymore!
  
  [Artificial] Neural Nets ***vaguely inspired*** by the connections we previously observed between the neurons of a brain. Initially, there probably was an intention to develop ANN to approximate biological brains. However, the modern working ANNs that we see their applications in various tasks are not designed to provide us a functional model of an animal brain. As far as I know, there is no study claiming they have found something new in a biological brain by looking into the connections and weight distributions of let's say a CNN or RNN model.
  
  
  ---
  
  ### Answer by username_4 on 2018-07-23. Score: 3
  
  The common statement that Artificial Neural Networks are inspired by the neural structure of brains is only partially true.
  
  It is true that Norbert Wiener, Claude Shannon, John von Neuman, and others began the path toward practical AI by developing what they then called the electronic brain. It is also true
  
  * Artificial networks have functions called activations,
  * Are wired in many-to-many relationships like biological neurons, and
  * Are designed to learn an optimal behavior,
  
  but that is the extent of the similarity. Cells in artificial networks such as MLPs (multilayer perceptrons) or RNN (Recurrent neural networks) are not like cells in brain networks.
  
  The perceptron, the first software stab at arrays of things that activate, was not an array of neurons. It was the application of basic feedback involving gradients, which had been in common use in engineering ever since James Watt's centrifugal governor was mathematically modeled by Gauss. Successive approximation, a principle that had been in use for centuries, was employed to incrementally update an attenuation matrix. The matrix was multiplied by the vector feeding an array of identical activation functions to produce output. That's it.
  
  The projection in a second dimension to a multi-layer topology was made possible by the realization that the Jacobian could be used to produce a corrective signal that, when distributed as negative feedback to the layers appropriately, could tune the attenuation matrix of a sequence of perceptrons and the network as a whole would converge upon satisfactory behavior. In the sequence of perceptrons, each element is called a layer. The feedback mechanism is now called back propagation.
  
  The mathematics used to correct the network is called gradient descent because it is like a dehydrated blind man using the gradient of the terrain to find water, and the issues of doing that are similar too. He might find a local minima (low point) before he finds fresh water and converge on death rather than hydration.
  
  The newer topologies are the additions of already existing convolution work used in digital image restoration, mail sorting, and graphics applications to create the CNN family of topologies and the ingenious use of what is like a chemical equilibrium from first year chemistry to combine optimization criteria creating the GAN family of topologies.
  
  Deep is simply a synonym for numerous in most AI contexts. It sometimes infers complexity in the higher level topology (above the vector-matrix products, the activations, and the convolutions).
  
  Active research is ongoing by those who are aware how different these deep networks are from what neural scientists have discovered decades ago in mammalian brain tissue. And there are more differentiators being discovered today as learning circuitry and neuro-chemistry in the brain is investigated from the genomic perspective.
  
  * Neural plasticity ... change in circuit topology due to dendrite and axiom growth, death, redirection, and other morphing
  * Topological complexity ... large number of axioms crisscross without interacting and are deliberately shielded from cross-talk (independent) most likely because it would be disadvantageous to let them connect [note 1]
  * Chemical signaling ... mammalian brains have dozens of neuro-transmitter and neuro-regulation compounds that have regional effects on circuitry [note 2]
  * Organelles ... living cells have many substructures and it is known that several types have complex relationships with signal transmission in neurons
  * Entirely different form of activation ... activations in common artificial neural nets are simply functions with ordinal scalars for both range and domain ... mammalian neurons operate as a function of both amplitude and relative temporal proximity of incoming signals [note 3]
  
  [1] Topology is ironically both a subset of architecture (in the fields of building design, network provisioning, WWW analysis, and semantic networks), yet at the same time topology is, much more than architecture, at the radical center of both AI mathematics and effective actualization in control systems
  
  [2] The role of chemistry may be essential to learning social and reproductive behavior that interrelates with DNA information propagation, linking in complex ways learning at the level of an ecosystem and the brain. Furthermore, long term and short term learning divides the brain's learning into two distinct capabilities too.
  
  [3] The impact of the timing of incoming signals on biological neuron activation is understood to some degree, but it may impact much more than neuron output. It may impact placticity and chemistry too, and the organelles may play a role in that.
  
  **Summary**
  
  What machine learning libraries do is as much simulating the human brain as Barbie and Ken dolls simulate a real couple.
  
  Nonetheless, remarkable things are arising in the field of deep learning, and it would not surprise me if autonomous vehicles become fully autonomous in our lifetimes. I would not recommend to any students to become a developer either. Computers will probably code much better than humans and orders of magnitude faster, and possibly soon. Some tasks are not of the kind that biology has evolved to do and computers can exceed human capabilities after only a few decades of research, eventually exceeding human performance by several orders of magnitude.
  
  > **Comments:**
  > **username_5**: There is a drastic difference between driving a car and programming a computer. Driving a car is a well-defined task, so there may be a way to express it as a function approximation problem, which is why neural networks may chime in. The function of using the car controls based on the input does exist and may be approximated. While programming a computer is not really a task, it is a sequence of decisions that one does while thinking about a situation. It is an art to turn troubles into problems.  
  
  ---
  
  