Answers.Body: <p>I think serali answered this question well, though I wanted to give some extra reading for those interested.</p>
  
  <p>There are many ways of deciphering what a neuron in a NN is doing. <a href="https://www.youtube.com/watch?v=6wcs6szJWMY" rel="nofollow noreferrer">This lecture</a> does a fantastic job at covering some of these methods and is an incredibly interesting watch. This covers more advanced methods of visualising what a model is doing.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-12-06T03:09:05.857Z
Answers.LastActivityDate: 2019-12-06T03:09:05.857Z
Answers.UserDisplayName: Recessive
Answers.Body: <p>In TensorFlow Playground, the horizontal line show where each class is separated for each neuron. What happens when you take any intermediate neuron to make the decision? You can see the answer by the line provided by that neuron. And this decision is a result of the weighted sum from the decisions of the previous neurons (up to activation).</p>
  
  <p>Take the middle-top neuron in the link you share, which is an almost horizontal line - slightly tilted to the right. This neuron classifies everything above it as a blue, and everything below it as an orange. Hover over the neuron to see a larger picture on the output. </p>
  
  <p>You can also see how this is actually calculated by hovering over the line coming from the neurons in the previous layer to the neuron you are looking at. For the case of the same neuron (center-top), the weight coming from the first input (<span class="math-container">$x_1$</span>) is 0.091, while from the second one (<span class="math-container">$x_2$</span>), it is 0.49. The neuron ends up being almost horizontal because the contribution from the horizontal input (<span class="math-container">$x_2$</span>) is so much larger compared to the vertical one (<span class="math-container">$x_1$</span>). </p>
  
  <p>Of course you need to take into account the nonlinearity coming from the activation function but the idea presented above is the essence of it. The example uses tanh activation, which behaves very linear in its intermediate region so one can ignore this issue to some extend for this particular case.  </p>
  
  <p>Edit: It appears that the values for the weights change at every browser session, so the neuron I describe might look a little different to you. To get the same configuration, simply click on the colored lines between neurons to edit them and use the values above for the connections. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-12-05T08:51:18.117Z
Answers.LastActivityDate: 2019-12-06T06:19:10.787Z
Answers.LastEditDate: 2019-12-06T06:19:10.787Z
Answers.UserDisplayName: serali
Body: <p>How can I know what each neuron does in NN?
  Consider the <a href="https://playground.tensorflow.org/" rel="nofollow noreferrer">Playground from Tensorflow</a>, there are some hidden layers with some neurons in each. Each of them shows a line(horizontal or vertical or ...). Where these shapes come from. I think they are understandable for nn not a person!</p>
  
Tags: <neural-networks><tensorflow>
Title: How can I find what does an specific neuron do in neural network?
UserDisplayName: Mahdi Amrollahi
ThreadText: # How can I find what does an specific neuron do in neural network?
  
  (Asked by: username_0 on 2019-12-05)
  
  How can I know what each neuron does in NN?
  Consider the [Playground from Tensorflow](https://playground.tensorflow.org/), there are some hidden layers with some neurons in each. Each of them shows a line(horizontal or vertical or ...). Where these shapes come from. I think they are understandable for nn not a person!
  
  **Tags:** neural-networks, tensorflow
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2019-12-05. Score: 3 | Accepted answer
  
  In TensorFlow Playground, the horizontal line show where each class is separated for each neuron. What happens when you take any intermediate neuron to make the decision? You can see the answer by the line provided by that neuron. And this decision is a result of the weighted sum from the decisions of the previous neurons (up to activation).
  
  Take the middle-top neuron in the link you share, which is an almost horizontal line - slightly tilted to the right. This neuron classifies everything above it as a blue, and everything below it as an orange. Hover over the neuron to see a larger picture on the output.
  
  You can also see how this is actually calculated by hovering over the line coming from the neurons in the previous layer to the neuron you are looking at. For the case of the same neuron (center-top), the weight coming from the first input ($x\_1$) is 0.091, while from the second one ($x\_2$), it is 0.49. The neuron ends up being almost horizontal because the contribution from the horizontal input ($x\_2$) is so much larger compared to the vertical one ($x\_1$).
  
  Of course you need to take into account the nonlinearity coming from the activation function but the idea presented above is the essence of it. The example uses tanh activation, which behaves very linear in its intermediate region so one can ignore this issue to some extend for this particular case.
  
  Edit: It appears that the values for the weights change at every browser session, so the neuron I describe might look a little different to you. To get the same configuration, simply click on the colored lines between neurons to edit them and use the values above for the connections.
  
  
  ---
  
  ### Answer by username_1 on 2019-12-06. Score: 1
  
  I think serali answered this question well, though I wanted to give some extra reading for those interested.
  
  There are many ways of deciphering what a neuron in a NN is doing. [This lecture](https://www.youtube.com/watch?v=6wcs6szJWMY) does a fantastic job at covering some of these methods and is an incredibly interesting watch. This covers more advanced methods of visualising what a model is doing.
  
  
  ---
  
  