Answers.Body: <p>Having no activation function means your activation function is the identity, namely <span class="math-container">$g(z)=z$</span>.</p>
  <p>Therefore, any derivative of <span class="math-container">$g(z)$</span> wrt to a parameter is simply the corresponding derivative of <span class="math-container">$z$</span>, with no extra factor.</p>
  <p>You would get the same result if you include the derivative, as it is multiplying by <span class="math-container">$1$</span>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-08-03T05:45:51.313Z
Answers.LastActivityDate: 2022-08-03T05:45:51.313Z
Answers.UserDisplayName: Jaume Oliver Lafont
Body: <p>I have a neural network that is composed of an input layer, two hidden layers and an output layer. The topology is [151, 200, 100, 1] I am using ReLU activation function on the neurons that are in the hidden layers and no activation function on the neuron that is in the output layer. I am wondering if when calculating the delta value of the neuron in the output layer, I should be using a derivative or if I should just subtract the expected output? Here I will put my line of code that this question concerns:</p>
  <pre><code>this.deltas[this.NN.length-2][0] = this.NN[this.NN.length-1][0] - expected;
  //In forward propagation this neuron has no activation function.
  </code></pre>
  
Tags: <neural-networks><backpropagation>
Title: In a neural network's neuron that has no activation function, to calculate the delta for the neuron during back propagation do you use a derivative?
UserDisplayName: Nick
ThreadText: # In a neural network's neuron that has no activation function, to calculate the delta for the neuron during back propagation do you use a derivative?
  
  (Asked by: username_0 on 2022-08-02)
  
  I have a neural network that is composed of an input layer, two hidden layers and an output layer. The topology is [151, 200, 100, 1] I am using ReLU activation function on the neurons that are in the hidden layers and no activation function on the neuron that is in the output layer. I am wondering if when calculating the delta value of the neuron in the output layer, I should be using a derivative or if I should just subtract the expected output? Here I will put my line of code that this question concerns:
  
  ```
  this.deltas[this.NN.length-2][0] = this.NN[this.NN.length-1][0] - expected;
  //In forward propagation this neuron has no activation function.
  
  ```
  
  **Tags:** neural-networks, backpropagation
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-08-03. Score: 2 | Accepted answer
  
  Having no activation function means your activation function is the identity, namely $g(z)=z$.
  
  Therefore, any derivative of $g(z)$ wrt to a parameter is simply the corresponding derivative of $z$, with no extra factor.
  
  You would get the same result if you include the derivative, as it is multiplying by $1$.
  
  
  ---
  
  