Answers.Body: <p>The architecture described in the paper is organized into three
  layers. The first layer contains 784 input neurons, the second layer
  <span class="math-container">$N$</span> excitatory neurons, and the third layer <span class="math-container">$N$</span> inhibitory neurons. In
  their S-sized network <span class="math-container">$N=200$</span>.</p>
  <p>The weight all synapses <em>from an input neuron</em> to its excitatory
  neurons decrease when it spikes. The weight of all synapses <em>to an
  excitatory neuron</em> from its input neurons increase when it
  spikes. Thus, spiking makes it easier for excitatory neurons to spike
  again when receiving stimuli on the same synapses.</p>
  <p>Meanwhile, the weight between excitatory and inhibitory neurons is so
  strong so that when an excitatory neuron spikes its inhibitory
  counterpart also spikes, depressing the other neurons and making it
  harder for them to spike. The winner neuron(s) slaps the loser neurons
  in the face.</p>
  <p>What I think you are missing is that it is all dependent on
  chance. Given some stimuli, <em>one</em> neuron will spike first and due to
  literal inhibition its advantage for that particular stimuli will grow
  larger and larger. That neuron will &quot;win&quot; that stimuli, while allowing
  the remaining neurons to compete for the remaining stimuli. To prevent
  one neuron from winning everything weights have to be normalized
  between samples.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-16T16:01:58.993Z
Answers.LastActivityDate: 2024-07-16T16:01:58.993Z
Answers.UserDisplayName: Gaslight Deceive Subvert
Body: <p>I stumbled upon a paper from P.Diehl and M.Cook with the title &quot;Unsupervised learning of digit recognition using spike-timing-dependent plasticity&quot; and I'm trying to understand the logic behind the network connection they made.</p>
  <p>The network is as follows. The inputs (size of an image 28x28) are connected to the usual all-to-all fashion with positive weights to an NxN layer of neurons. They are decoded using poisson random distribution in which the frequency of spikes of a pixel is set accordingly to the pixel value. The NxN layer is connected 1 on 1 to an NxN layer of inhibitory neurons. These neurons inhibit all other neurons except the one they are connected to. Thus they are connected all-to-all with an exception.</p>
  <p>According to the paper this provides competition among neurons. I cannot understand how, in this particular connection, competition is provided. How can different neurons inherit different properties? To me it seems that all neurons will inherit the same properties, thus no differences in weights will be made in the training session among all neurons. For example, if the input 5 is passed to the network all weights of all neurons will try to adjust according to 5. Then if input 7 is passed next, all the weights will be updated according to the new number (7). It is expected, though, that some weights will keep the previous adjustment ie that some weights will have the properties of 5 and the others the properties of 7.</p>
  
Tags: <spiking-neural-networks>
Title: How does Lateral Inhibition Provide Competition among Neurons?
UserDisplayName: Cand3
ThreadText: # How does Lateral Inhibition Provide Competition among Neurons?
  
  (Asked by: username_0 on 2020-06-27)
  
  I stumbled upon a paper from P.Diehl and M.Cook with the title "Unsupervised learning of digit recognition using spike-timing-dependent plasticity" and I'm trying to understand the logic behind the network connection they made.
  
  The network is as follows. The inputs (size of an image 28x28) are connected to the usual all-to-all fashion with positive weights to an NxN layer of neurons. They are decoded using poisson random distribution in which the frequency of spikes of a pixel is set accordingly to the pixel value. The NxN layer is connected 1 on 1 to an NxN layer of inhibitory neurons. These neurons inhibit all other neurons except the one they are connected to. Thus they are connected all-to-all with an exception.
  
  According to the paper this provides competition among neurons. I cannot understand how, in this particular connection, competition is provided. How can different neurons inherit different properties? To me it seems that all neurons will inherit the same properties, thus no differences in weights will be made in the training session among all neurons. For example, if the input 5 is passed to the network all weights of all neurons will try to adjust according to 5. Then if input 7 is passed next, all the weights will be updated according to the new number (7). It is expected, though, that some weights will keep the previous adjustment ie that some weights will have the properties of 5 and the others the properties of 7.
  
  **Tags:** spiking-neural-networks
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-07-16. Score: 0
  
  The architecture described in the paper is organized into three
  layers. The first layer contains 784 input neurons, the second layer
  $N$ excitatory neurons, and the third layer $N$ inhibitory neurons. In
  their S-sized network $N=200$.
  
  The weight all synapses *from an input neuron* to its excitatory
  neurons decrease when it spikes. The weight of all synapses *to an
  excitatory neuron* from its input neurons increase when it
  spikes. Thus, spiking makes it easier for excitatory neurons to spike
  again when receiving stimuli on the same synapses.
  
  Meanwhile, the weight between excitatory and inhibitory neurons is so
  strong so that when an excitatory neuron spikes its inhibitory
  counterpart also spikes, depressing the other neurons and making it
  harder for them to spike. The winner neuron(s) slaps the loser neurons
  in the face.
  
  What I think you are missing is that it is all dependent on
  chance. Given some stimuli, *one* neuron will spike first and due to
  literal inhibition its advantage for that particular stimuli will grow
  larger and larger. That neuron will "win" that stimuli, while allowing
  the remaining neurons to compete for the remaining stimuli. To prevent
  one neuron from winning everything weights have to be normalized
  between samples.
  
  
  ---
  
  