<p>I think what you mean to ask is <b>how can differentiation occur when there's no obvious neural network function to differentiate?</b></p>

<p>Don't worry - lots of people get confused about this, because it seems like an obvious hole in the puzzle. As mentioned by @AtillaOzgur, neural networks use partial differentiation through backpropagation. </p>

<p>First, take the output of all the neurons (except the one you're about to differentiate by) as a function:</p>

<p><a href="https://i.sstatic.net/jIrA9.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/jIrA9.png" alt="enter image description here"></a></p>

<p>The above diagram represents the output of one neuron. Do this for every neuron in your network until you have a set. Let's call this set function NN. The output of NN (given all your neuron outputs) is what you'd normally plug into your RL policy.</p>

<p>You then <b>differentiate NN by a single neuron (n) as shown:</b></p>

<p><span class="math-container">$$\frac{\partial NN}{\partial n} = \lim_{h\to0} \left(\frac{NN(\text{all other neuron outputs}, n + h) - NN(\text{all other neuron outputs}, n)}{h} \right)$$</span></p>

<p>In reality however, it's the partial derivative of the activation function (A) with respect to the output of a single neuron (n):</p>

<p><span class="math-container">$$\frac{\partial A}{\partial n}$$</span></p>

<p>So, depending on your activation function, you just plug in your neuron output to a certain expression and you've found the value by which to update your neural network.</p>

<p>I hope this helps. Deep learning is definitely a field with a learning curve, but places like StackExchange are great resources to help you out.</p>

CC BY-SA 4.0
2019-09-12T17:16:58.063Z
So, when we talk about the derivative of a NN w.r.t a weight, we are actually refering to the change in the output of the NN caused by the change of one neuron( to which that weight is associated). Can you explain a bit more why "In reality however, it's the partial derivative of the activation function (A) with respect to the output of a single neuron (n)"? Thanks.
qiang li
CC BY-SA 4.0
2019-09-12T17:21:12.753Z
@qiangli Sure- differentiating the activation function is a mathematical shortcut to computing the weight of each neuron. Instead of plugging everything into the neural net to iterate over again and again, we can activate the result and get a gradient per-neuron. This is possible through some matrix calculus and algebraic rearranging: it's all available online, but learning the rearrangement probably won't deepen your understanding.
mcRobusta
CC BY-SA 4.0
2019-09-12T14:13:25.160Z
2019-09-13T22:04:39.023Z
2019-09-13T22:04:39.023Z
mcRobusta
<p>Neural networks (NNs) are used as approximators in reinforcement learning (RL). To update the policy in RL, the actor network's gradients w.r.t its weights are needed. Since NN doesn't have a mathematical expression to work with, how can its derivatives be calculated?</p>

CC BY-SA 4.0
2019-09-12T11:20:51.947Z
What do you mean Neural Networks does not have a mathmatical expression to work with? Neural Networks is purely math in its nature?
SandMan
<neural-networks><reinforcement-learning><backpropagation><actor-critic-methods><policy-gradients>
How can the derivative of a neural network be calculated, given no mathematical expression?
qiang li
# How can the derivative of a neural network be calculated, given no mathematical expression?

(Asked by: username_0 on 2019-09-12)

Neural networks (NNs) are used as approximators in reinforcement learning (RL). To update the policy in RL, the actor network's gradients w.r.t its weights are needed. Since NN doesn't have a mathematical expression to work with, how can its derivatives be calculated?

**Tags:** neural-networks, reinforcement-learning, backpropagation, actor-critic-methods, policy-gradients

> **Comments:**
> **username_1**: What do you mean Neural Networks does not have a mathmatical expression to work with? Neural Networks is purely math in its nature?  

---

## 1 answers

### Answer by username_2 on 2019-09-12. Score: 5 | Accepted answer

I think what you mean to ask is **how can differentiation occur when there's no obvious neural network function to differentiate?**

Don't worry - lots of people get confused about this, because it seems like an obvious hole in the puzzle. As mentioned by @AtillaOzgur, neural networks use partial differentiation through backpropagation.

First, take the output of all the neurons (except the one you're about to differentiate by) as a function:

[![enter image description here](https://i.sstatic.net/jIrA9.png)](https://i.sstatic.net/jIrA9.png)

The above diagram represents the output of one neuron. Do this for every neuron in your network until you have a set. Let's call this set function NN. The output of NN (given all your neuron outputs) is what you'd normally plug into your RL policy.

You then **differentiate NN by a single neuron (n) as shown:**

$$\frac{\partial NN}{\partial n} = \lim\_{h\to0} \left(\frac{NN(\text{all other neuron outputs}, n + h) - NN(\text{all other neuron outputs}, n)}{h} \right)$$

In reality however, it's the partial derivative of the activation function (A) with respect to the output of a single neuron (n):

$$\frac{\partial A}{\partial n}$$

So, depending on your activation function, you just plug in your neuron output to a certain expression and you've found the value by which to update your neural network.

I hope this helps. Deep learning is definitely a field with a learning curve, but places like StackExchange are great resources to help you out.

> **Comments:**
> **username_0**: So, when we talk about the derivative of a NN w.r.t a weight, we are actually refering to the change in the output of the NN caused by the change of one neuron( to which that weight is associated). Can you explain a bit more why "In reality however, it's the partial derivative of the activation function (A) with respect to the output of a single neuron (n)"? Thanks.  
> **username_2**: @qiangli Sure- differentiating the activation function is a mathematical shortcut to computing the weight of each neuron. Instead of plugging everything into the neural net to iterate over again and again, we can activate the result and get a gradient per-neuron. This is possible through some matrix calculus and algebraic rearranging: it's all available online, but learning the rearrangement probably won't deepen your understanding.  

---

