Answers.Body: <p>Even when it is always said that &quot;neural network cells&quot; are &quot;inspired&quot; in biological neurons, there are critical differences that made this similitude only an &quot;inspiration&quot;. Thus, comparison of a CNN or any other kind of NN with brain is always a fuzzy comparison.</p>
  <p>The biggest difference is probably the learning capacity: the human brain learns by itself, while the neural network needs an <strong>external system</strong> (the back-propagation algorithm) that feeds the NN with the network parameters (the knowledge).</p>
  <p>In other words: a neural network, understood as the set of nodes that execute a basic function (lets say weighted addition and activation function) is <strong>absolutely unable to learn anything by itself</strong>. The knowledge is written in the neural network as a set of weights and other parameters. This is not an intelligent process, as it is not load a program in a conventional computer. The system who learns is the system who runs the back-propagation algorithm and <strong>this system is not a neural net</strong>.</p>
  <p>In comparison, the human mind, understood as a network of cells,is able to learn by itself from the experience.</p>
  <blockquote>
  <p>Thus, when we train our supervised classifier so that it updates
  weights thanks to backpropagation algorithm (whose philosophy is the
  changing of weights to minimize an error function that characterizes
  the comparison between the network output and the ground truth), are
  we applying a sort of Hebb's Rule and, more properly, the concept of
  synaptic plasticity since weights change during the learning process?</p>
  </blockquote>
  <p>Absolutely no.</p>
  <p>A learning process based in Hebb's rule will:</p>
  <ul>
  <li>not need an external system to run the back-propagation algorithm</li>
  <li>increase the weight between two cells when an artificial neuron and the next one triggers more or less at same time.</li>
  </ul>
  <p>This is not the method of the back-propagation algorithm:</p>
  <ul>
  <li>the back-propagation algorithm doesn't execute in the neural net</li>
  <li>the back-propagation algorithm compares expected results and network ones and, using math calculus, modifies the network weights.</li>
  </ul>
  <p>Nowadays, I do not know of any system based on biological neuron behavior and able to learn. Just a few models as been able to learn the eigenvector of a basic process. Even worst, no body as been able to replicate basic behavior of well know biological brains as the one of c-elegans worm, a brain that is completely mapped in number of neurons and synapses between them.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-02-11T20:26:48.493Z
Answers.LastActivityDate: 2022-02-11T20:38:42.297Z
Answers.LastEditDate: 2022-02-11T20:38:42.297Z
Answers.UserDisplayName: pasaba por aqui
Body: <p>Is there some of Hebb's rule behind the concept of backpropagation learning rule of a simple supervised neural network, that for example is trained for classification task ?</p>
  <p>I was reading about the concept of <em>synaptic plasticity</em> that is explained in simple words here <a href="https://qbi.uq.edu.au/brain-basics/brain/brain-physiology/what-synaptic-plasticity" rel="nofollow noreferrer">https://qbi.uq.edu.au/</a>:</p>
  <blockquote>
  <p><strong>Plasticity is the ability of the brain to change and adapt to new information. Synaptic plasticity is change that occurs at synapses</strong>, the junctions between neurons that allow them to communicate.
  The idea that synapses could change, and that this change depended on how active or inactive they were, was first proposed in the 1949 by Canadian psychologist Donald Hebb.</p>
  </blockquote>
  <p>And more in particular I found in <a href="https://en.wikipedia.org/wiki/Hebbian_theory#:%7E:text=Hebbian%20theory%20is%20a%20neuroscientific,stimulation%20of%20a%20postsynaptic%20cell.&amp;text=The%20theory%20is%20also%20called,postulate%2C%20and%20cell%20assembly%20theory." rel="nofollow noreferrer">Wikipedia</a> that:</p>
  <blockquote>
  <p>Hebbian theory is a neuroscientific theory claiming that an <strong>increase in synaptic efficacy</strong> <strong>arises from</strong> a presynaptic cell's <strong>repeated and persistent stimulation</strong> of a postsynaptic cell. It is an attempt to explain synaptic plasticity, <em><strong>the adaptation of brain neurons during the learning process.</strong></em></p>
  </blockquote>
  <p>In an artificial neural network, synaptic efficacy (<em>that is the strength of communication between neurons</em>, from <a href="https://www.frontiersin.org/articles/10.3389/fncel.2017.00367/full#:%7E:text=Synaptic%20efficacy%20refers%20to%20the,number%20of%20postsynaptic%20receptors%20activated.&amp;text=Several%20electrophysiological%20methods%20are%20widely%20used%20to%20monitor%20synaptic%20transmission." rel="nofollow noreferrer">https://www.frontiersin.org</a>) depends on weights that are associated to connections between neurons (from what I understood reading the beginning of paragraph <em>44.3 The basic unit — the neuron</em> in <em>Handbook of Chemometrics and Qualimetrics: Part B</em> <a href="https://www.sciencedirect.com/topics/computer-science/synaptic-strength" rel="nofollow noreferrer">https://www.sciencedirect.com</a>).</p>
  <p>Thus, when we train our supervised classifier so that it updates weights thanks to backpropagation algorithm (whose philosophy is the changing of weights to minimize an error function that characterizes the comparison between the network output and the ground truth), are we applying a sort of Hebb's Rule and, more properly, the concept of <em>synaptic plasticity since weights change during the learning process</em>?</p>
  <p>Or Hebb's rule means only &quot;Fire together, wire together&quot; as I understand by reading the answer to this SE post <a href="https://ai.stackexchange.com/questions/2531/how-do-you-explain-hebbian-learning-in-an-intuitive-way">How do you explain Hebbian Learning in an intuitive way?</a>.</p>
  <p>I read also this other SE answer to the post <a href="https://ai.stackexchange.com/questions/12011/is-there-a-hebb-neural-network">Is there a Hebb neural network?</a> in which it is clearly explained that neural networks (or models) that can learn in a Hebbian fashion are different from those based on backpropagation algorithm.</p>
  <p>But if synaptic plasticity means a changing in the connections between neurons (synapses), that in artifical neural networks means a changing in the weights, during a learning process, can I say that also neural networks based on back-propagation algorithms exploit the concept of synaptic plasticity (and so the Hebb's rule) ?</p>
  
Tags: <neural-networks><training><backpropagation><biology><hebbian-learning>
Title: Do neural networks, trained with backpropagation algorithm, exploit the concept of synaptic plasticity?
UserDisplayName: HelpNeederStudent
ThreadText: # Do neural networks, trained with backpropagation algorithm, exploit the concept of synaptic plasticity?
  
  (Asked by: username_0 on 2022-02-10)
  
  Is there some of Hebb's rule behind the concept of backpropagation learning rule of a simple supervised neural network, that for example is trained for classification task ?
  
  I was reading about the concept of *synaptic plasticity* that is explained in simple words here [https://qbi.uq.edu.au/](https://qbi.uq.edu.au/brain-basics/brain/brain-physiology/what-synaptic-plasticity):
  
  > **Plasticity is the ability of the brain to change and adapt to new information. Synaptic plasticity is change that occurs at synapses**, the junctions between neurons that allow them to communicate.
  > The idea that synapses could change, and that this change depended on how active or inactive they were, was first proposed in the 1949 by Canadian psychologist Donald Hebb.
  
  And more in particular I found in [Wikipedia](https://en.wikipedia.org/wiki/Hebbian_theory#:%7E:text=Hebbian%20theory%20is%20a%20neuroscientific,stimulation%20of%20a%20postsynaptic%20cell.&text=The%20theory%20is%20also%20called,postulate%2C%20and%20cell%20assembly%20theory.) that:
  
  > Hebbian theory is a neuroscientific theory claiming that an **increase in synaptic efficacy** **arises from** a presynaptic cell's **repeated and persistent stimulation** of a postsynaptic cell. It is an attempt to explain synaptic plasticity, ***the adaptation of brain neurons during the learning process.***
  
  In an artificial neural network, synaptic efficacy (*that is the strength of communication between neurons*, from [https://www.frontiersin.org](https://www.frontiersin.org/articles/10.3389/fncel.2017.00367/full#:%7E:text=Synaptic%20efficacy%20refers%20to%20the,number%20of%20postsynaptic%20receptors%20activated.&text=Several%20electrophysiological%20methods%20are%20widely%20used%20to%20monitor%20synaptic%20transmission.)) depends on weights that are associated to connections between neurons (from what I understood reading the beginning of paragraph *44.3 The basic unit — the neuron* in *Handbook of Chemometrics and Qualimetrics: Part B* [https://www.sciencedirect.com](https://www.sciencedirect.com/topics/computer-science/synaptic-strength)).
  
  Thus, when we train our supervised classifier so that it updates weights thanks to backpropagation algorithm (whose philosophy is the changing of weights to minimize an error function that characterizes the comparison between the network output and the ground truth), are we applying a sort of Hebb's Rule and, more properly, the concept of *synaptic plasticity since weights change during the learning process*?
  
  Or Hebb's rule means only "Fire together, wire together" as I understand by reading the answer to this SE post [How do you explain Hebbian Learning in an intuitive way?](https://ai.stackexchange.com/questions/2531/how-do-you-explain-hebbian-learning-in-an-intuitive-way).
  
  I read also this other SE answer to the post [Is there a Hebb neural network?](https://ai.stackexchange.com/questions/12011/is-there-a-hebb-neural-network) in which it is clearly explained that neural networks (or models) that can learn in a Hebbian fashion are different from those based on backpropagation algorithm.
  
  But if synaptic plasticity means a changing in the connections between neurons (synapses), that in artifical neural networks means a changing in the weights, during a learning process, can I say that also neural networks based on back-propagation algorithms exploit the concept of synaptic plasticity (and so the Hebb's rule) ?
  
  **Tags:** neural-networks, training, backpropagation, biology, hebbian-learning
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-02-11. Score: 2 | Accepted answer
  
  Even when it is always said that "neural network cells" are "inspired" in biological neurons, there are critical differences that made this similitude only an "inspiration". Thus, comparison of a CNN or any other kind of NN with brain is always a fuzzy comparison.
  
  The biggest difference is probably the learning capacity: the human brain learns by itself, while the neural network needs an **external system** (the back-propagation algorithm) that feeds the NN with the network parameters (the knowledge).
  
  In other words: a neural network, understood as the set of nodes that execute a basic function (lets say weighted addition and activation function) is **absolutely unable to learn anything by itself**. The knowledge is written in the neural network as a set of weights and other parameters. This is not an intelligent process, as it is not load a program in a conventional computer. The system who learns is the system who runs the back-propagation algorithm and **this system is not a neural net**.
  
  In comparison, the human mind, understood as a network of cells,is able to learn by itself from the experience.
  
  > Thus, when we train our supervised classifier so that it updates
  > weights thanks to backpropagation algorithm (whose philosophy is the
  > changing of weights to minimize an error function that characterizes
  > the comparison between the network output and the ground truth), are
  > we applying a sort of Hebb's Rule and, more properly, the concept of
  > synaptic plasticity since weights change during the learning process?
  
  Absolutely no.
  
  A learning process based in Hebb's rule will:
  
  * not need an external system to run the back-propagation algorithm
  * increase the weight between two cells when an artificial neuron and the next one triggers more or less at same time.
  
  This is not the method of the back-propagation algorithm:
  
  * the back-propagation algorithm doesn't execute in the neural net
  * the back-propagation algorithm compares expected results and network ones and, using math calculus, modifies the network weights.
  
  Nowadays, I do not know of any system based on biological neuron behavior and able to learn. Just a few models as been able to learn the eigenvector of a basic process. Even worst, no body as been able to replicate basic behavior of well know biological brains as the one of c-elegans worm, a brain that is completely mapped in number of neurons and synapses between them.
  
  
  ---
  
  