Answers.Body: <p>Formally speaking <span class="math-container">$x_6$</span> is a function of <span class="math-container">$w_{16},\ w_{26}$</span> and <span class="math-container">$w_{36}$</span>, that is <span class="math-container">$$x_6 =f(w_{16}, w_{26}, w_{36})=w_{16}y_1 + w_{26}y_2 + w_{36}y_3.$$</span>
  The derivative w.r.t. <span class="math-container">$w_{26}$</span> is
  <span class="math-container">$$\frac{\partial x_6}{\partial w_{26}}= \frac{\partial w_{16}y_1}{\partial w_{26}} +\frac{\partial w_{26}y_2}{\partial w_{26}}  +\frac{\partial w_{36}y_3}{\partial w_{26}} = 0 +y_2 \frac{\partial w_{26}}{\partial w_{26}} + 0= y_2.$$</span>
  The first equality is obtained using the fact that the partial derivative is linear (so the derivative of the sum is the sum of the derivatives); the second equality comes from again from the linearity and from the fact that <span class="math-container">$w_{16}y_1$</span> and <span class="math-container">$w_{36}y_3$</span> are constants with respect to <span class="math-container">$w_{26}$</span>, so their partial derivative w.r.t. this variable is <span class="math-container">$0$</span>.</p>
  <p><em><strong>Bonus</strong></em></p>
  <p>Not really asked in the original question, but since I'm here let me have a bit of fun ;).</p>
  <p>Let's say <span class="math-container">$x_6$</span> is the output of the sixth node after you apply an activation function, that is <span class="math-container">$$x_6 =\sigma(f(w_{16}, w_{26}, w_{36}))=\sigma(w_{16}y_1 + w_{26}y_2 + w_{36}y_3).$$</span>
  You can compute the partial derivative applying the properties illustrated above, with the additional help of the chain rule
  <span class="math-container">$$\frac{\partial x_6}{\partial w_{26}}=\frac{\partial \sigma(w_{16}y_1 + w_{26}y_2 + w_{36}y_3)}{\partial w_{26}}=\sigma'\frac{\partial w_{16}y_1}{\partial w_{26}} +\sigma'\frac{\partial w_{26}y_2}{\partial w_{26}}  +\sigma'\frac{\partial w_{36}y_3}{\partial w_{26}}=y_2\sigma'$$</span>
  <span class="math-container">$\sigma'$</span> denotes the derivative of sigma with respect to its argument.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-04T19:36:42.503Z
Answers.LastActivityDate: 2021-01-04T22:42:20.240Z
Answers.LastEditDate: 2021-01-04T22:42:20.240Z
Answers.UserDisplayName: Uskebasi
Body: <p>If I have a neural network, and say the 6th output node of the neural network is:</p>
  <p><span class="math-container">$$x_6 = w_{16}y_1 + w_{26}y_2 + w_{36}y_3$$</span></p>
  <p>What does that make the derivative of:</p>
  <p><span class="math-container">$$\frac{\partial x_6}{\partial w_{26}}$$</span></p>
  <p>I guess that it's how is <span class="math-container">$x_6$</span> changing with respect to <span class="math-container">$w_{26}$</span>, so, therefore, is it equal to <span class="math-container">$y_2$</span> (since the output, <span class="math-container">$y_2$</span>, will change depending on the weight added to the input)?</p>
  
Tags: <neural-networks><gradient-descent><calculus>
Title: What is the derivative of a specific output with respect to a specific weight?
UserDisplayName: Slowat_Kela
ThreadText: # What is the derivative of a specific output with respect to a specific weight?
  
  (Asked by: username_0 on 2021-01-02)
  
  If I have a neural network, and say the 6th output node of the neural network is:
  
  $$x\_6 = w\_{16}y\_1 + w\_{26}y\_2 + w\_{36}y\_3$$
  
  What does that make the derivative of:
  
  $$\frac{\partial x\_6}{\partial w\_{26}}$$
  
  I guess that it's how is $x\_6$ changing with respect to $w\_{26}$, so, therefore, is it equal to $y\_2$ (since the output, $y\_2$, will change depending on the weight added to the input)?
  
  **Tags:** neural-networks, gradient-descent, calculus
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-01-04. Score: 2 | Accepted answer
  
  Formally speaking $x\_6$ is a function of $w\_{16},\ w\_{26}$ and $w\_{36}$, that is $$x\_6 =f(w\_{16}, w\_{26}, w\_{36})=w\_{16}y\_1 + w\_{26}y\_2 + w\_{36}y\_3.$$
  The derivative w.r.t. $w\_{26}$ is
  $$\frac{\partial x\_6}{\partial w\_{26}}= \frac{\partial w\_{16}y\_1}{\partial w\_{26}} +\frac{\partial w\_{26}y\_2}{\partial w\_{26}} +\frac{\partial w\_{36}y\_3}{\partial w\_{26}} = 0 +y\_2 \frac{\partial w\_{26}}{\partial w\_{26}} + 0= y\_2.$$
  The first equality is obtained using the fact that the partial derivative is linear (so the derivative of the sum is the sum of the derivatives); the second equality comes from again from the linearity and from the fact that $w\_{16}y\_1$ and $w\_{36}y\_3$ are constants with respect to $w\_{26}$, so their partial derivative w.r.t. this variable is $0$.
  
  ***Bonus***
  
  Not really asked in the original question, but since I'm here let me have a bit of fun ;).
  
  Let's say $x\_6$ is the output of the sixth node after you apply an activation function, that is $$x\_6 =\sigma(f(w\_{16}, w\_{26}, w\_{36}))=\sigma(w\_{16}y\_1 + w\_{26}y\_2 + w\_{36}y\_3).$$
  You can compute the partial derivative applying the properties illustrated above, with the additional help of the chain rule
  $$\frac{\partial x\_6}{\partial w\_{26}}=\frac{\partial \sigma(w\_{16}y\_1 + w\_{26}y\_2 + w\_{36}y\_3)}{\partial w\_{26}}=\sigma'\frac{\partial w\_{16}y\_1}{\partial w\_{26}} +\sigma'\frac{\partial w\_{26}y\_2}{\partial w\_{26}} +\sigma'\frac{\partial w\_{36}y\_3}{\partial w\_{26}}=y\_2\sigma'$$
  $\sigma'$ denotes the derivative of sigma with respect to its argument.
  
  
  ---
  
  