Answers.Body: <ol>
  <li>The correct formula for updating the weights between the hidden layer and the output layer is:
  <span class="math-container">$$\Delta W_{j,k} = h_k \ \cdot \ o'_{j} \ \cdot \ (o_j - t_j),$$</span>
  where <span class="math-container">$h$</span> is the activated hidden layer and <span class="math-container">$o'$</span> is the derivative of output layer.<br />
  I found this formula in the book <strong>Artificial Intelligence: A Guide to Intelligent Systems</strong> by <strong>Michael Negnevitsky</strong>.</li>
  </ol>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-05-06T21:24:36.790Z
Answers.LastActivityDate: 2021-05-06T21:24:36.790Z
Answers.UserDisplayName: Tiago Cavalcante
Body: <p>I'm creating a neural network with 3 layers and no bias.</p>
  <p>On internet I saw that the expression for the derivative of the weights between the hidden layer and the output layer was:</p>
  <p><span class="math-container">$$\Delta W_{j,k} = (o_k - t_k) \cdot f'\left[\sum_j (W_{j,k} \ \cdot o_j)\right] \cdot o_j,$$</span></p>
  <p>where <span class="math-container">$t$</span> is the target output, <span class="math-container">$o$</span> is the activated output layer and <span class="math-container">$f'$</span> the derivative of the activation function.</p>
  <p>But the shape of these weights is <span class="math-container">$\text{output nodes}\times\text{hidden nodes}$</span>, and <span class="math-container">$\text{hidden nodes}$</span> can be bigger than <span class="math-container">$\text{output nodes}$</span>, so the formula is wrong because of I'm taking <span class="math-container">$o_k$</span> and <span class="math-container">$o$</span> has length <span class="math-container">$\text{output nodes}$</span>.</p>
  <ol>
  <li><p>In simple terms, what is the right formula for updating these weights?</p>
  </li>
  <li><p>Also, what is the right formula for updating the weights between the input layer and the hidden layer?</p>
  </li>
  </ol>
  
Tags: <neural-networks><machine-learning><backpropagation><calculus>
Title: What is the correct formula for updating the weights in a 1-single hidden layer neural network?
UserDisplayName: Tiago Cavalcante
ThreadText: # What is the correct formula for updating the weights in a 1-single hidden layer neural network?
  
  (Asked by: username_0 on 2021-05-06)
  
  I'm creating a neural network with 3 layers and no bias.
  
  On internet I saw that the expression for the derivative of the weights between the hidden layer and the output layer was:
  
  $$\Delta W\_{j,k} = (o\_k - t\_k) \cdot f'\left[\sum\_j (W\_{j,k} \ \cdot o\_j)\right] \cdot o\_j,$$
  
  where $t$ is the target output, $o$ is the activated output layer and $f'$ the derivative of the activation function.
  
  But the shape of these weights is $\text{output nodes}\times\text{hidden nodes}$, and $\text{hidden nodes}$ can be bigger than $\text{output nodes}$, so the formula is wrong because of I'm taking $o\_k$ and $o$ has length $\text{output nodes}$.
  
  1. In simple terms, what is the right formula for updating these weights?
  2. Also, what is the right formula for updating the weights between the input layer and the hidden layer?
  
  **Tags:** neural-networks, machine-learning, backpropagation, calculus
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2021-05-06. Score: 1
  
  1. The correct formula for updating the weights between the hidden layer and the output layer is:
     $$\Delta W\_{j,k} = h\_k \ \cdot \ o'\_{j} \ \cdot \ (o\_j - t\_j),$$
     where $h$ is the activated hidden layer and $o'$ is the derivative of output layer.  
     I found this formula in the book **Artificial Intelligence: A Guide to Intelligent Systems** by **Michael Negnevitsky**.
  
  
  ---
  
  