Answers.Body: <blockquote>
    <p>Does the back propagation algorithm modifies the weigh values and bias values in the same pass?</p>
  </blockquote>
  
  <p>Yes.</p>
  
  <blockquote>
    <p>How does the algorithm decide whether it has to change the weight value or bias value to reduce the error in a pass?</p>
  </blockquote>
  
  <p>It differentiates the loss function (like MSE) with respect to the weights and biases, that is, it finds the <em>partial derivative</em> of the loss function with respect to each of the parameters. You can think of the partial derivative of the loss function with respect to one of the parameters of the model as representing the "contribution" of that parameter to the loss of the model. </p>
  
  <p>The partial derivatives of the loss function with respect to each of the parameters (weights and biases) is collectively called the <em>gradient</em>, which is thus a vector of <span class="math-container">$N$</span> partial derivatives, where <span class="math-container">$N$</span> is the number of parameters of the model.</p>
  
  <blockquote>
    <p>Will the learning rate same for bias and weights?</p>
  </blockquote>
  
  <p>It is usually the same, but, in theory, nobody prevents you from updating the biases using a different learning rate, so you could use a different learning rate to update the biases and weights. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-08T16:51:59.377Z
Answers.LastActivityDate: 2019-04-08T16:57:43.260Z
Answers.LastEditDate: 2019-04-08T16:57:43.260Z
Answers.UserDisplayName: nbro
Body: <p>I am new to deep learning. I have doubts on modifying bias values during back propagation. My doubts are</p>
  
  <ol>
  <li>Does the back propagation algorithm modifies the weigh values and bias values in the same pass?</li>
  <li>How does the algorithm decide whether it has to change the weight value or bias value to reduce the error in a pass?</li>
  <li>Will the learning rate same for bias and weights? </li>
  </ol>
  
  <p>Thanks!</p>
  
Tags: <deep-learning><convolutional-neural-networks><backpropagation><gradient-descent>
Title: When is bias values updated in back propagation?
UserDisplayName: Maanu
ThreadText: # When is bias values updated in back propagation?
  
  (Asked by: username_0 on 2019-04-08)
  
  I am new to deep learning. I have doubts on modifying bias values during back propagation. My doubts are
  
  1. Does the back propagation algorithm modifies the weigh values and bias values in the same pass?
  2. How does the algorithm decide whether it has to change the weight value or bias value to reduce the error in a pass?
  3. Will the learning rate same for bias and weights?
  
  Thanks!
  
  **Tags:** deep-learning, convolutional-neural-networks, backpropagation, gradient-descent
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-04-08. Score: 2 | Accepted answer
  
  > Does the back propagation algorithm modifies the weigh values and bias values in the same pass?
  
  Yes.
  
  > How does the algorithm decide whether it has to change the weight value or bias value to reduce the error in a pass?
  
  It differentiates the loss function (like MSE) with respect to the weights and biases, that is, it finds the *partial derivative* of the loss function with respect to each of the parameters. You can think of the partial derivative of the loss function with respect to one of the parameters of the model as representing the "contribution" of that parameter to the loss of the model.
  
  The partial derivatives of the loss function with respect to each of the parameters (weights and biases) is collectively called the *gradient*, which is thus a vector of $N$ partial derivatives, where $N$ is the number of parameters of the model.
  
  > Will the learning rate same for bias and weights?
  
  It is usually the same, but, in theory, nobody prevents you from updating the biases using a different learning rate, so you could use a different learning rate to update the biases and weights.
  
  
  ---
  
  