Answers.Body: <p>The basic calculation for a single neuron is of the form</p>
  
  <p><span class="math-container">$$\sigma\left(\sum_{i} x_i  w_i \right),$$</span></p>
  
  <p>where <span class="math-container">$x_i$</span> is the input to the neuron <span class="math-container">$w_i$</span> are the neuron-specific weights for <em>every single</em> input and <span class="math-container">$\sigma$</span> is the pre-specified activation function. In your terms, and disregarding the activation function, the calculation would turn out to be</p>
  
  <p><span class="math-container">$$c\,a_c + d\,a_d + b$$</span></p>
  
  <p>Note, that the bias term <span class="math-container">$b$</span> is just a weight that gets multiplied by the input <span class="math-container">$1$</span>, thus it appears to have no input.</p>
  
  <p>If you want to develop a further understanding for this, you should try to get familiar with matrix and vector notations and the basic linear algebra that underlies the <em>feed-forward neural networks</em>. If you do, an entire layer of neurons on a whole batch of data will suddenly simply look like this:</p>
  
  <p><span class="math-container">$$\sigma(WX)$$</span></p>
  
  <p>and a FFNN with say 3 layers will look like this:</p>
  
  <p><span class="math-container">$$\sigma_{3}(W_3\sigma_2(W_2\sigma_1(W_1X)))$$</span></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-19T14:55:08.970Z
Answers.Text: I mean in multiple layers ANN, if two neurons (in the same layer) is pointing the same neuron (at the next layer), what will the input of the neuron that's pointed be? Is it the addition of each arrow pointing it? Or the multiply of each arrow pointing it.
Answers.UserDisplayName: Andrew.Wolphoe
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-19T14:56:37.490Z
Answers.Text: Also ANN doesn't require addition? (So you use y=as instead of y=ax+b?) Or addition is only required in LSTM?
Answers.UserDisplayName: Andrew.Wolphoe
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-19T15:23:24.293Z
Answers.Text: I can see that you are still confused, so I'll try to explain it as well. Let's take for example an input of shape 3 X=[1,2,3] and now if we only have one neuron we will need 3 weights, one for each input right? Lets say W=[4,5,6]. When we multiply the input with the weights we will get 1*4 + 2*5 + 3*6 = 32, and that's the output of the first neuron. Now lets say that we want to add another neuron, instead of 3 weights we will need 6. W=[[4,5,6].[7,8,9]]. The output of the first neuron is the same, but the second one will output 1*7 + 2*8 + 3*9.
Answers.UserDisplayName: razvanc92
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-19T22:37:24.033Z
Answers.Text: @razvanc92 I'm talking at ANN with depth>1. So look at [this picture](https://i.sstatic.net/zKUJc.jpg), the bottom section are only multiplying (only one weight for each neuron and top section are following input*a+b=output. And my question are written in the bottom right of each section
Answers.UserDisplayName: Andrew.Wolphoe
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-19T14:50:52.710Z
Answers.LastActivityDate: 2019-07-19T17:46:32.367Z
Answers.LastEditDate: 2019-07-19T17:46:32.367Z
Answers.UserDisplayName: Simon Krannig
Answers.Body: <p>Simon Krannig's answer provides the math notation behined exactly what is going on, but since you still seem a bit confused, I've made a visual representation of a neural network using <em>only weights with no activation function</em>. See below:
  <a href="https://i.sstatic.net/vkQkI.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/vkQkI.png" alt="enter image description here"></a></p>
  
  <p>So I'm fairly sure it as you suspected: At each neuron, you take the <strong>sum</strong> of the inputs of the previous layer multiplied by the weight that connects that specific input to said neuron, where each input has its own unique weight for every one of its outgoing connections.</p>
  
  <p>With a bias, you would do the exact same math as shown in the above image, but once you find the final value (0.2, -0.15, 0.16 and -0.075, the output layer doesn't have a bias) you would add the bias to the total value. So see below for an example including a bias:</p>
  
  <p><a href="https://i.sstatic.net/Tyxsy.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Tyxsy.png" alt="enter image description here"></a></p>
  
  <p><strong>NOTE</strong> I did not update the outputs at each layer to include the bias because I can't be bothered redrawing this in paint. Just know that the final value for all the nodes with the brown bias haven't carried over to the next layer.</p>
  
  <p>Then, if you were to include an activation function, you would finally take your value and put it through. So including the bias', looking at node 1 of layer 2, it would be (lets pretend your activation function is a sigmoid):</p>
  
  <pre><code>sigmoid((0.4*0.5)+0.2)
  </code></pre>
  
  <p>and for layer 3 node 2:</p>
  
  <pre><code>sigmoid(((0.6*0.2)+(1.3*-0.15))-0.4)
  </code></pre>
  
  <p>That is how you would do a forward pass of a simple neural network.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-22T03:57:49.357Z
Answers.LastActivityDate: 2019-07-22T03:57:49.357Z
Answers.UserDisplayName: Recessive
Body: <p>To me, most ANN/RNN related articles don't tell me actually how the network is implemented. I know that in the ANN you'll have multiple neurons, activation function, weights, etc. But, how do you, actually, in each neuron, convert the input to the output?</p>
  
  <p>Putting activation function aside, is the neuron simply doing <span class="math-container">$\text{input}*a+b=\text{output}$</span>, and try to find the correct <span class="math-container">$a$</span> and <span class="math-container">$b$</span>? If it's true, then how about where you have two neurons and their output (<span class="math-container">$c$</span> and <span class="math-container">$d$</span>) is pointing to one neuron? Do you first multiply <span class="math-container">$c$</span> and <span class="math-container">$d$</span> then feed it in as input?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-19T11:04:52.703Z
Comments.Text: Could you give the maths or pseudo-code  for "Do you first multiply c and d then feed it in as input?" e.g. Do you mean (using your previous example) `c*d*a + b = output`? Note that I, and probably many others here, would call that pseudo-code, not maths, although clearly the two things are related. The maths might look like 
  $y = ax + b$ and be concise for large networks if you are happy for all the variables to be vectors or matrices
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-19T11:12:42.447Z
Comments.Text: Yeah I mean (c*d)*a+b=output though the a,b might not be the same as the last sample as they're different neuron
Comments.UserDisplayName: Andrew.Wolphoe
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-19T14:57:29.890Z
Comments.Text: @NeilSlater What I'm asking is why the input is consider c*d (if it is) not things like (c+d)?
Comments.UserDisplayName: Andrew.Wolphoe
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-19T17:21:03.967Z
Comments.Text: Yes I understand generally what you are asking, I wanted to clarify what you thought the formula might be, because an English description of a maths equation is not precise. Whilst the equation (or code) is precise. You have an answer now, and it looks correct to me. You *could* write a NN based on multiplying inputs together (and it may even be good at some things), but Simon's answer gives the typical formula for a basic modern NN
Comments.UserDisplayName: Neil Slater
Tags: <neural-networks><recurrent-neural-networks><artificial-neuron><neurons>
Title: How do layers in an artificial neural network transform inputs to outputs?
UserDisplayName: Andrew.Wolphoe
ThreadText: # How do layers in an artificial neural network transform inputs to outputs?
  
  (Asked by: username_0 on 2019-07-19)
  
  To me, most ANN/RNN related articles don't tell me actually how the network is implemented. I know that in the ANN you'll have multiple neurons, activation function, weights, etc. But, how do you, actually, in each neuron, convert the input to the output?
  
  Putting activation function aside, is the neuron simply doing $\text{input}\*a+b=\text{output}$, and try to find the correct $a$ and $b$? If it's true, then how about where you have two neurons and their output ($c$ and $d$) is pointing to one neuron? Do you first multiply $c$ and $d$ then feed it in as input?
  
  **Tags:** neural-networks, recurrent-neural-networks, artificial-neuron, neurons
  
  > **Comments:**
  > **username_1**: Could you give the maths or pseudo-code  for "Do you first multiply c and d then feed it in as input?" e.g. Do you mean (using your previous example) `c*d*a + b = output`? Note that I, and probably many others here, would call that pseudo-code, not maths, although clearly the two things are related. The maths might look like  $y = ax + b$ and be concise for large networks if you are happy for all the variables to be vectors or matrices  
  > **username_0**: Yeah I mean (c*d)*a+b=output though the a,b might not be the same as the last sample as they're different neuron  
  > **username_0**: @username_1 What I'm asking is why the input is consider c*d (if it is) not things like (c+d)?  
  > **username_1**: Yes I understand generally what you are asking, I wanted to clarify what you thought the formula might be, because an English description of a maths equation is not precise. Whilst the equation (or code) is precise. You have an answer now, and it looks correct to me. You *could* write a NN based on multiplying inputs together (and it may even be good at some things), but username_2's answer gives the typical formula for a basic modern NN  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_4 on 2019-07-22. Score: 2 | Accepted answer
  
  username_2's answer provides the math notation behined exactly what is going on, but since you still seem a bit confused, I've made a visual representation of a neural network using *only weights with no activation function*. See below:
  [![enter image description here](https://i.sstatic.net/vkQkI.png)](https://i.sstatic.net/vkQkI.png)
  
  So I'm fairly sure it as you suspected: At each neuron, you take the **sum** of the inputs of the previous layer multiplied by the weight that connects that specific input to said neuron, where each input has its own unique weight for every one of its outgoing connections.
  
  With a bias, you would do the exact same math as shown in the above image, but once you find the final value (0.2, -0.15, 0.16 and -0.075, the output layer doesn't have a bias) you would add the bias to the total value. So see below for an example including a bias:
  
  [![enter image description here](https://i.sstatic.net/Tyxsy.png)](https://i.sstatic.net/Tyxsy.png)
  
  **NOTE** I did not update the outputs at each layer to include the bias because I can't be bothered redrawing this in paint. Just know that the final value for all the nodes with the brown bias haven't carried over to the next layer.
  
  Then, if you were to include an activation function, you would finally take your value and put it through. So including the bias', looking at node 1 of layer 2, it would be (lets pretend your activation function is a sigmoid):
  
  ```
  sigmoid((0.4*0.5)+0.2)
  
  ```
  
  and for layer 3 node 2:
  
  ```
  sigmoid(((0.6*0.2)+(1.3*-0.15))-0.4)
  
  ```
  
  That is how you would do a forward pass of a simple neural network.
  
  
  ---
  
  ### Answer by username_2 on 2019-07-19. Score: 4
  
  The basic calculation for a single neuron is of the form
  
  $$\sigma\left(\sum\_{i} x\_i w\_i \right),$$
  
  where $x\_i$ is the input to the neuron $w\_i$ are the neuron-specific weights for *every single* input and $\sigma$ is the pre-specified activation function. In your terms, and disregarding the activation function, the calculation would turn out to be
  
  $$c\,a\_c + d\,a\_d + b$$
  
  Note, that the bias term $b$ is just a weight that gets multiplied by the input $1$, thus it appears to have no input.
  
  If you want to develop a further understanding for this, you should try to get familiar with matrix and vector notations and the basic linear algebra that underlies the *feed-forward neural networks*. If you do, an entire layer of neurons on a whole batch of data will suddenly simply look like this:
  
  $$\sigma(WX)$$
  
  and a FFNN with say 3 layers will look like this:
  
  $$\sigma\_{3}(W\_3\sigma\_2(W\_2\sigma\_1(W\_1X)))$$
  
  > **Comments:**
  > **username_0**: I mean in multiple layers ANN, if two neurons (in the same layer) is pointing the same neuron (at the next layer), what will the input of the neuron that's pointed be? Is it the addition of each arrow pointing it? Or the multiply of each arrow pointing it.  
  > **username_0**: Also ANN doesn't require addition? (So you use y=as instead of y=ax+b?) Or addition is only required in LSTM?  
  > **username_3**: I can see that you are still confused, so I'll try to explain it as well. Let's take for example an input of shape 3 X=[1,2,3] and now if we only have one neuron we will need 3 weights, one for each input right? Lets say W=[4,5,6]. When we multiply the input with the weights we will get 1*4 + 2*5 + 3*6 = 32, and that's the output of the first neuron. Now lets say that we want to add another neuron, instead of 3 weights we will need 6. W=[[4,5,6].[7,8,9]]. The output of the first neuron is the same, but the second one will output 1*7 + 2*8 + 3*9.  
  > **username_0**: @username_3 I'm talking at ANN with depth>1. So look at [this picture](https://i.sstatic.net/zKUJc.jpg), the bottom section are only multiplying (only one weight for each neuron and top section are following input*a+b=output. And my question are written in the bottom right of each section  
  
  ---
  
  