<p>The change from <span class="math-container">$c$</span> as a decision variable towards <span class="math-container">$k'$</span> as a decision variable is by a simple change of variables.</p>
<p>In the original setup, <span class="math-container">$k$</span> is the state and <span class="math-container">$c$</span> is the control (decision) variable. The Bellman equation is the following.
<span class="math-container">$$
v(k) = \max_{c \in [0, Ak^\alpha]} u(c) + \beta v(Ak^\alpha - c).
$$</span>
(here the constraint <span class="math-container">$k' = Ak^\alpha - c$</span> is already substituted into the Bellman equation).</p>
<p>Now, you can define <span class="math-container">$k' = Ak^\alpha - c$</span>. As <span class="math-container">$c \in [0, Ak^\alpha]$</span>, we have that <span class="math-container">$k' \in [0, Ak^\alpha]$</span>. Notice that there is a one-to-one correspondence between the values of <span class="math-container">$c$</span> and the values of <span class="math-container">$k'$</span>. Substituting <span class="math-container">$c$</span> by <span class="math-container">$Ak^\alpha - k'$</span> in the Bellman equation gives:
<span class="math-container">$$
v(k) = \max_{k' \in [0, Ak^\alpha]} u(Ak^\alpha - k') + \beta v(k').
$$</span>
In this problem <span class="math-container">$k$</span> is the state and <span class="math-container">$k'$</span> is the control (and also the state in the next period.</p>
<p>The Envelope condition gives:
<span class="math-container">$$
v'(k) = A \alpha k^{\alpha - 1} u'(Ak^\alpha - k')
$$</span>
The first order condition gives:
<span class="math-container">$$
-u'(Ak^\alpha - k') + \beta v'(k') = 0.
$$</span>
These two can be combined to give the desired Euler equation.</p>
<p><strong>General setting</strong></p>
<p>In general, the Bellman equation takes the following form:
<span class="math-container">$$
v(x) = \max_{u \in \Gamma(x)} f(x,u) + \beta v(r(x,u)).
$$</span>
where <span class="math-container">$x$</span> is the state variable and <span class="math-container">$u$</span> is the control. The law-of-motion function <span class="math-container">$x' = r(x,u)$</span> gives the state tomorrow <span class="math-container">$(x')$</span> as a function of the state today <span class="math-container">$(x)$</span> and the control today <span class="math-container">$(u)$</span>.</p>
<p>If <span class="math-container">$r$</span> is invertible in <span class="math-container">$u$</span> (as a function of <span class="math-container">$u$</span>) we can sometimes rewrite this as:
<span class="math-container">$$
v(x) = \max_{x' \in \Delta(x)} g(x,x') + \beta v(x').
$$</span>
where <span class="math-container">$g(x,x') = f(x, r^{-1}(x,x'))$</span> and
<span class="math-container">$$
\Delta(x) = \{x'| \exists u \in \Gamma(x), x' = r(x,u)\}.
$$</span></p>
<p>This new Bellman equation has as state <span class="math-container">$(x)$</span> and control <span class="math-container">$(x')$</span> which happen to coincide with the state tomorrow.</p>

CC BY-SA 4.0
2023-07-16T08:18:18.857Z
2023-07-16T08:18:18.857Z
tdm
<p>I am trying to better understand the process of deriving Euler Equations using
the first order condition of the problem on the right hand side of a Bellman equation
and the Benveniste-Scheinkman formula. In particular, there is a line in Ljungqvist and Sargent's (LS) Recursive Macroeconomic Theory that I don't quite understand in practice. It is this:</p>
<blockquote>
<p>When the state and controls can be defined in such a way that only <span class="math-container">$u$</span> appears in the transition equation, i.e., <span class="math-container">$x' = g(u)$</span>: the derivative of the value function becomes,...
<span class="math-container">$$V'(x)=r_1(x,h(x)).$$</span></p>
</blockquote>
<p>I see why the result follows, granted that states and controls can be defined in the way described. But lets look at a simple optimal growth example.</p>
<p>Let the sequential problem be</p>
<p><span class="math-container">$$\max\sum_{t=0}^\infty\beta^t\log(c_t) \\
\text{s.t.} \\
k_{t+1} + c_t = Ak_t^\alpha.$$</span></p>
<p>The associated Bellman equation is
<span class="math-container">$$V(k)=\max\log(c) + \beta V(k') \\
\text{s.t.} \\
k' + c = Ak^\alpha.$$</span></p>
<p>Now, in my current understanding <span class="math-container">$k'$</span> and <span class="math-container">$c$</span> are both controls and <span class="math-container">$k$</span> is the state. But, (SL) say &quot;let the state be <span class="math-container">$k$</span> and the control be <span class="math-container">$k'$</span> where <span class="math-container">$k'$</span> denotes next period's value of <span class="math-container">$k$</span>.&quot; However, this does not give me the desired form, i.e. writing
<span class="math-container">$k'=Ak^\alpha-c$</span> has both a control and a state on the right hand side. How does this fit into the form <span class="math-container">$x'=g(u)$</span>?</p>

<macroeconomics><dynamic-programming><recursive-macroeconomics>
Regarding the arbitrariness of states and controls
ConsistentlyCaffeinated
# Regarding the arbitrariness of states and controls

(Asked by: username_0 on 2023-07-13)

I am trying to better understand the process of deriving Euler Equations using
the first order condition of the problem on the right hand side of a Bellman equation
and the Benveniste-Scheinkman formula. In particular, there is a line in Ljungqvist and Sargent's (LS) Recursive Macroeconomic Theory that I don't quite understand in practice. It is this:

> When the state and controls can be defined in such a way that only $u$ appears in the transition equation, i.e., $x' = g(u)$: the derivative of the value function becomes,...
> $$V'(x)=r\_1(x,h(x)).$$

I see why the result follows, granted that states and controls can be defined in the way described. But lets look at a simple optimal growth example.

Let the sequential problem be

$$\max\sum\_{t=0}^\infty\beta^t\log(c\_t) \\
\text{s.t.} \\
k\_{t+1} + c\_t = Ak\_t^\alpha.$$

The associated Bellman equation is
$$V(k)=\max\log(c) + \beta V(k') \\
\text{s.t.} \\
k' + c = Ak^\alpha.$$

Now, in my current understanding $k'$ and $c$ are both controls and $k$ is the state. But, (SL) say "let the state be $k$ and the control be $k'$ where $k'$ denotes next period's value of $k$." However, this does not give me the desired form, i.e. writing
$k'=Ak^\alpha-c$ has both a control and a state on the right hand side. How does this fit into the form $x'=g(u)$?

**Tags:** macroeconomics, dynamic-programming, recursive-macroeconomics


---

## 1 answers

### Answer by username_1 on 2023-07-16. Score: 1

The change from $c$ as a decision variable towards $k'$ as a decision variable is by a simple change of variables.

In the original setup, $k$ is the state and $c$ is the control (decision) variable. The Bellman equation is the following.
$$
v(k) = \max\_{c \in [0, Ak^\alpha]} u(c) + \beta v(Ak^\alpha - c).
$$
(here the constraint $k' = Ak^\alpha - c$ is already substituted into the Bellman equation).

Now, you can define $k' = Ak^\alpha - c$. As $c \in [0, Ak^\alpha]$, we have that $k' \in [0, Ak^\alpha]$. Notice that there is a one-to-one correspondence between the values of $c$ and the values of $k'$. Substituting $c$ by $Ak^\alpha - k'$ in the Bellman equation gives:
$$
v(k) = \max\_{k' \in [0, Ak^\alpha]} u(Ak^\alpha - k') + \beta v(k').
$$
In this problem $k$ is the state and $k'$ is the control (and also the state in the next period.

The Envelope condition gives:
$$
v'(k) = A \alpha k^{\alpha - 1} u'(Ak^\alpha - k')
$$
The first order condition gives:
$$
-u'(Ak^\alpha - k') + \beta v'(k') = 0.
$$
These two can be combined to give the desired Euler equation.

**General setting**

In general, the Bellman equation takes the following form:
$$
v(x) = \max\_{u \in \Gamma(x)} f(x,u) + \beta v(r(x,u)).
$$
where $x$ is the state variable and $u$ is the control. The law-of-motion function $x' = r(x,u)$ gives the state tomorrow $(x')$ as a function of the state today $(x)$ and the control today $(u)$.

If $r$ is invertible in $u$ (as a function of $u$) we can sometimes rewrite this as:
$$
v(x) = \max\_{x' \in \Delta(x)} g(x,x') + \beta v(x').
$$
where $g(x,x') = f(x, r^{-1}(x,x'))$ and
$$
\Delta(x) = \{x'| \exists u \in \Gamma(x), x' = r(x,u)\}.
$$

This new Bellman equation has as state $(x)$ and control $(x')$ which happen to coincide with the state tomorrow.


---

