<p>In a sense, the Bellman equation is a definition of <span class="math-container">$V(k)$</span>, which might be more obvious if it's written
<span class="math-container">$$
V(k):=\max_{0\leq k'\leq f\left(k\right)}u\left(f\left(k\right)-k'\right)+\beta V\left(k'\right).
$$</span>
The operator <span class="math-container">$\max_{0\leq k'\leq f\left(k\right)}$</span> tells us that, to get <span class="math-container">$V(k)$</span>, we should find the <em>specific</em> <span class="math-container">$k'$</span> - let's call it <span class="math-container">$k'^*$</span> - at which the expression <span class="math-container">$u\left(f\left(k\right)-k'\right)+\beta V\left(k'\right)$</span> is greatest (if such a maximum exists).</p>
<p>To do that, and provided the functions involved have the right properties, the first step is to take the derivative of <span class="math-container">$u\left(f\left(k\right)-k'\right)+\beta V\left(k'\right)$</span> with respect to <span class="math-container">$k'$</span> and setting the result equal to zero. So,
<span class="math-container">$$
\frac{d}{dk'}u\left(f\left(k\right)-k'\right)+\beta V\left(k'\right) = 0, 
$$</span>
from which follows (after some math) that
<span class="math-container">$$
u'\left(f\left(k\right)-k'\right)=\beta V'\left(k'\right)
$$</span>
<em>at the maximum</em> (given the right properties), so not everywhere, but where <span class="math-container">$k'=k'^*$</span>.
Without further knowledge of <span class="math-container">$u$</span> and <span class="math-container">$V$</span>, this gives at least an implicit rule for how to obtain <span class="math-container">$k'^*$</span>, namely via
<span class="math-container">$$
u'\left(f\left(k\right)-k'^*\right)=\beta V'\left(k'^*\right)
$$</span>
Because with <span class="math-container">$k'^*$</span> we have now found the specific <span class="math-container">$k'$</span> at which <span class="math-container">$u'\left(f\left(k\right)-k'\right)=\beta V'\left(k'\right)$</span> is greatest, we can substitute that specific value, <span class="math-container">$k'^*$</span>, into the initial definition to get
<span class="math-container">$$
V(k):=u\left(f\left(k\right)-k'^*\right)+\beta V\left(k'^*\right).
$$</span>
where the <span class="math-container">$\max$</span> operator has disappeared, as <span class="math-container">$k'^*$</span> evaluates the function already at the max. And we see that <span class="math-container">$V(k)$</span> does not depend on <span class="math-container">$k'$</span>.</p>

CC BY-SA 4.0
2022-08-25T23:23:18.843Z
Would it be correct to write V(k) as actually V(k,k') and then differentiate both sides? I will accept this answer, if you could please write down explicitly what we are differentiating? In other  words, when we take the derivative, does the max operator disappear?
Kwame Brown
CC BY-SA 4.0
2022-08-26T07:52:52.817Z
@KwameBrown $V$ only depends on $k$, not on $k'$. For a given $k$, we only care about the specific value of $k'$ where the inner function is maximized. The rest is calculus, to find a (local) maxima of a (differentiable) function, compute its derivative and set it to zero (and check a few more things).
quarague
CC BY-SA 4.0
2022-08-26T08:22:43.617Z
Answer edited for clarification.
BrsG
CC BY-SA 4.0
2022-08-26T09:03:51.503Z
+1, nice answer
Robbinspire
CC BY-SA 4.0
2022-08-26T12:58:24.643Z
Thank you so much, this is very helpful indeed.
Kwame Brown
CC BY-SA 4.0
2022-08-25T22:07:35.177Z
2022-08-26T08:34:26.880Z
2022-08-26T08:34:26.880Z
BrsG
<p>Assume that we have a Bellman equation that is</p>
<p><span class="math-container">$$
V(k)=\max_{0\leq k'\leq f\left(k\right)}u\left(f\left(k\right)-k'\right)+\beta V\left(k'\right)
$$</span>
The textbook says that if we differentiate with respect to <span class="math-container">$k'$</span> and
set equal to <span class="math-container">$0,$</span> we get:
<span class="math-container">$$
u'\left(f\left(k\right)-k'\right)=\beta V'\left(k'\right)
$$</span></p>
<p>My questions are:  i) What are we differentiating here? <span class="math-container">$V(k)$</span>? Also, how can we differentiate under the max operator?</p>

<bellman-equations>
Differentiating Bellman equation
Kwame Brown
# Differentiating Bellman equation

(Asked by: username_0 on 2022-08-25)

Assume that we have a Bellman equation that is

$$
V(k)=\max\_{0\leq k'\leq f\left(k\right)}u\left(f\left(k\right)-k'\right)+\beta V\left(k'\right)
$$
The textbook says that if we differentiate with respect to $k'$ and
set equal to $0,$ we get:
$$
u'\left(f\left(k\right)-k'\right)=\beta V'\left(k'\right)
$$

My questions are: i) What are we differentiating here? $V(k)$? Also, how can we differentiate under the max operator?

**Tags:** bellman-equations


---

## 1 answers

### Answer by username_1 on 2022-08-25. Score: 5 | Accepted answer

In a sense, the Bellman equation is a definition of $V(k)$, which might be more obvious if it's written
$$
V(k):=\max\_{0\leq k'\leq f\left(k\right)}u\left(f\left(k\right)-k'\right)+\beta V\left(k'\right).
$$
The operator $\max\_{0\leq k'\leq f\left(k\right)}$ tells us that, to get $V(k)$, we should find the *specific* $k'$ - let's call it $k'^\*$ - at which the expression $u\left(f\left(k\right)-k'\right)+\beta V\left(k'\right)$ is greatest (if such a maximum exists).

To do that, and provided the functions involved have the right properties, the first step is to take the derivative of $u\left(f\left(k\right)-k'\right)+\beta V\left(k'\right)$ with respect to $k'$ and setting the result equal to zero. So,
$$
\frac{d}{dk'}u\left(f\left(k\right)-k'\right)+\beta V\left(k'\right) = 0,
$$
from which follows (after some math) that
$$
u'\left(f\left(k\right)-k'\right)=\beta V'\left(k'\right)
$$
*at the maximum* (given the right properties), so not everywhere, but where $k'=k'^\*$.
Without further knowledge of $u$ and $V$, this gives at least an implicit rule for how to obtain $k'^\*$, namely via
$$
u'\left(f\left(k\right)-k'^\*\right)=\beta V'\left(k'^\*\right)
$$
Because with $k'^\*$ we have now found the specific $k'$ at which $u'\left(f\left(k\right)-k'\right)=\beta V'\left(k'\right)$ is greatest, we can substitute that specific value, $k'^\*$, into the initial definition to get
$$
V(k):=u\left(f\left(k\right)-k'^\*\right)+\beta V\left(k'^\*\right).
$$
where the $\max$ operator has disappeared, as $k'^\*$ evaluates the function already at the max. And we see that $V(k)$ does not depend on $k'$.

> **Comments:**
> **username_0**: Would it be correct to write V(k) as actually V(k,k') and then differentiate both sides? I will accept this answer, if you could please write down explicitly what we are differentiating? In other  words, when we take the derivative, does the max operator disappear?  
> **username_2**: @username_0 $V$ only depends on $k$, not on $k'$. For a given $k$, we only care about the specific value of $k'$ where the inner function is maximized. The rest is calculus, to find a (local) maxima of a (differentiable) function, compute its derivative and set it to zero (and check a few more things).  
> **username_1**: Answer edited for clarification.  
> **username_3**: +1, nice answer  
> **username_0**: Thank you so much, this is very helpful indeed.  

---

