<p>You work in discrete time, see for example <a href="http://www.princeton.edu/~moll/ECO521Web/Lecture5_ECO521_web.pdf" rel="nofollow">this resource</a>.  </p>

<p>Or <a href="http://www.princeton.edu/~moll/HACT.pdf" rel="nofollow">this (incomplete) paper</a>, where the same author participates. In the beginning the paper talks about "coarse discrete methods" and may give the impression that it contains some continuous time simulation method. It really doesn't -see the description of their algorithm in the Appendix, page 30.  </p>

<p>Also <a href="https://people.maths.ox.ac.uk/witte/JHWitte_DPhilThesis_13_March_2012.pdf." rel="nofollow">this MPhil dissertation</a>, which, being a graduate thesis, describes the steps in some detail, and with a wealth of referenced literature.</p>

CC BY-SA 3.0
2014-11-19T19:56:10.833Z
Both Ben's notes and his paper don't really mention how to simulate them, only how to solve them - and that's not my problem..
FooBar
CC BY-SA 3.0
2014-11-19T19:25:38.357Z
2014-11-19T19:25:38.357Z
Alecos Papadopoulos
<p>Since Ben's notes were already mentioned I'm not quite sure why you don't iterate over time using a forward approximation as he suggests
\begin{align}
\dot k(t) :\approx \frac{k(t+1) - k(t)}{\Delta t} \Leftrightarrow k(t+1) \approx \Delta t \dot k(t) + k(t)
\end{align} </p>

<p>which gives 
\begin{align}
k(1) \approx \Delta t (z-c(0)) + k(0)
\end{align}</p>

CC BY-SA 3.0
2015-09-10T18:16:02.063Z
You can also look at the approximations made with finite difference equation. It is a good method do discretize a continous time model.
optimal control
CC BY-SA 3.0
2015-09-10T17:43:05.573Z
2015-09-10T17:43:05.573Z
clueless
<p>Once you have solved the equation $\rho V(k) = \max_c g(c) + V'(k)(z - c)$
to find the function $V(k)$ you should be able to solve for what is called the policy function. The function $V(k)$ is the value function. Knowing $V(k)$ should then give you the
information that you need to solve for the policy, say $c(k,t)$. This will then tell you how  $k$ will evolve over time.</p>

CC BY-SA 3.0
2014-11-19T22:06:57.953Z
I know $c(k,t)$, my problem is how to understand it. How *exactly* do I get $k(1)$, given that I know the policy function and $k(0)$?
FooBar
CC BY-SA 3.0
2014-11-19T21:50:13.933Z
2014-11-19T21:50:13.933Z
jmbejara
<p>Expanding on jmbejara's answer and your question about integration, there are really three steps here:</p>

<ol>
<li>Solve the HJB equation $\rho V(k) = \max_c g(c)+V'(k)(z-c)$. (You're already done this.)</li>
<li>Obtain the policy function $c(k)=\text{argmax}_c g(c)+V'(k)(z-c)$. (You presumably already did this while solving the HJB.)</li>
<li>Substitute $c(k)$ into the law of motion for $k$ and numerically solve that differential equation for $k(t)$:$$\dot{k}=z-c(k)$$</li>
</ol>

<p>It sounds like you already had (1) and (2) and were just unsure about what exactly to do in (3). If you're working in a deterministic environment (and from the HJB it looks like you are), no discretization on your part is necessary: you can just use a canned ODE solver from Matlab or any other software package to numerically solve the differential equation above for $k(t)$.</p>

<p>To be a little more specific: </p>

<ul>
<li>If the HJB is simple and you have an analytical solution for $V(k)$, and thus can easily compute $c(k)$ as well, that's great. You can just plug $c(k)$ into the law of motion for $k$ and tell your software package of choice to solve it and obtain the trajectory of $k(t)$.</li>
<li>If you had to numerically solve the HJB, then that's great because it means that you probably have already used some kind of numerical ODE solution technique and will be familiar with it. On the minus side, it means that when you initially obtain $V(k)$ and $c(k)$, you'll only get it for a grid of $k$, rather than having some kind of formula you can apply everywhere - which is a problem, because you'll need to be able to determine $c(k)$ at any point $k$ in a continuum if you're going to numerically solve the $\dot{k}$ differential equation for $k(t)$. The simplest way around this, to my knowledge, is to fit a spline to (or use some other approximation method on) the discrete grid of $(k_{grid},c(k_{grid}))$ values that you have. Then you can obtain an approximate value of $c(k)$ anywhere that's covered by the spline, and it will be extremely accurate assuming that your ODE is well-behaved and you have a dense grid.</li>
</ul>

<p>Things get more complicated in a stochastic environment, but your case looks deterministic. (And to be clear, I'm just a dabbler here: numerical methods are not my area of expertise. But this is what I know.)</p>

CC BY-SA 3.0
2014-12-14T20:09:57.893Z
2014-12-14T20:09:57.893Z
nominally rigid
<p>Say I have solved an HJB of the form:</p>

<p>$\rho V(k) = \max_c g(c) + V'(k)(z - c)$</p>

<p>I have calibrated $\rho$ to monthly parameters. I would like to simulate the development of $k$. I start with $k(0)$. However, unlike in discrete-time, I'm not sure what happens next.</p>

<p>Is $k(1) = k(0) + (z-c(0))$, with $k(1)$ being the value of $k$ for the next month? Or do I need to somehow integrate along the lines of $k(1) = \int_0^1 (z-c(t))dt + k(0)$? There's many references on solving these things, but I couldn't find any references on simulating it.</p>

CC BY-SA 3.0
2014-11-19T18:16:06.347Z
I believe it would be helpful if you could include in the question's title the full name, prior to use the initials HJB. "Hamilton-Jacobi-Bellman" may not come immediately to mind, to those browsing the list of question titles.
Alecos Papadopoulos
CC BY-SA 3.0
2014-11-20T16:35:05.750Z
I nearly always work in discrete time for macro, for this reason. That said, I've written simulations for continuous-time, non-macro econ models. When I've done that, the approach was to decide on a $\Delta$t step size, change all equations to difference equations (decide some timing issues), and then simulate. Unfortunately I have no immediate public examples of this for you -- and unsure it actually answers your question (hence the comment instead of an answer).
CompEcon
CC BY-SA 3.0
2014-11-20T16:36:47.000Z
@Foobar: Have you tried the integration method you suggested? How did that go if you did?
CompEcon
CC BY-SA 3.0
2014-11-20T16:39:37.987Z
Well, it succeeds in giving me a number. It's hard to actually falsify or validate the result.
FooBar
<mathematical-economics><simulations><computation>
Simulating a Hamilton-Jacobi-Bellman
FooBar
# Simulating a Hamilton-Jacobi-Bellman

(Asked by: username_0 on 2014-11-19)

Say I have solved an HJB of the form:

$\rho V(k) = \max\_c g(c) + V'(k)(z - c)$

I have calibrated $\rho$ to monthly parameters. I would like to simulate the development of $k$. I start with $k(0)$. However, unlike in discrete-time, I'm not sure what happens next.

Is $k(1) = k(0) + (z-c(0))$, with $k(1)$ being the value of $k$ for the next month? Or do I need to somehow integrate along the lines of $k(1) = \int\_0^1 (z-c(t))dt + k(0)$? There's many references on solving these things, but I couldn't find any references on simulating it.

**Tags:** mathematical-economics, simulations, computation

> **Comments:**
> **username_1**: I believe it would be helpful if you could include in the question's title the full name, prior to use the initials HJB. "Hamilton-Jacobi-Bellman" may not come immediately to mind, to those browsing the list of question titles.  
> **username_2**: I nearly always work in discrete time for macro, for this reason. That said, I've written simulations for continuous-time, non-macro econ models. When I've done that, the approach was to decide on a $\Delta$t step size, change all equations to difference equations (decide some timing issues), and then simulate. Unfortunately I have no immediate public examples of this for you -- and unsure it actually answers your question (hence the comment instead of an answer).  
> **username_2**: @username_0: Have you tried the integration method you suggested? How did that go if you did?  
> **username_0**: Well, it succeeds in giving me a number. It's hard to actually falsify or validate the result.  

---

## 4 answers

### Answer by username_6 on 2014-12-14. Score: 4 | Accepted answer

Expanding on username_5's answer and your question about integration, there are really three steps here:

1. Solve the HJB equation $\rho V(k) = \max\_c g(c)+V'(k)(z-c)$. (You're already done this.)
2. Obtain the policy function $c(k)=\text{argmax}\_c g(c)+V'(k)(z-c)$. (You presumably already did this while solving the HJB.)
3. Substitute $c(k)$ into the law of motion for $k$ and numerically solve that differential equation for $k(t)$:$$\dot{k}=z-c(k)$$

It sounds like you already had (1) and (2) and were just unsure about what exactly to do in (3). If you're working in a deterministic environment (and from the HJB it looks like you are), no discretization on your part is necessary: you can just use a canned ODE solver from Matlab or any other software package to numerically solve the differential equation above for $k(t)$.

To be a little more specific:

* If the HJB is simple and you have an analytical solution for $V(k)$, and thus can easily compute $c(k)$ as well, that's great. You can just plug $c(k)$ into the law of motion for $k$ and tell your software package of choice to solve it and obtain the trajectory of $k(t)$.
* If you had to numerically solve the HJB, then that's great because it means that you probably have already used some kind of numerical ODE solution technique and will be familiar with it. On the minus side, it means that when you initially obtain $V(k)$ and $c(k)$, you'll only get it for a grid of $k$, rather than having some kind of formula you can apply everywhere - which is a problem, because you'll need to be able to determine $c(k)$ at any point $k$ in a continuum if you're going to numerically solve the $\dot{k}$ differential equation for $k(t)$. The simplest way around this, to my knowledge, is to fit a spline to (or use some other approximation method on) the discrete grid of $(k\_{grid},c(k\_{grid}))$ values that you have. Then you can obtain an approximate value of $c(k)$ anywhere that's covered by the spline, and it will be extremely accurate assuming that your ODE is well-behaved and you have a dense grid.

Things get more complicated in a stochastic environment, but your case looks deterministic. (And to be clear, I'm just a dabbler here: numerical methods are not my area of expertise. But this is what I know.)


---

### Answer by username_1 on 2014-11-19. Score: 2

You work in discrete time, see for example [this resource](http://www.princeton.edu/~moll/ECO521Web/Lecture5_ECO521_web.pdf).

Or [this (incomplete) paper](http://www.princeton.edu/~moll/HACT.pdf), where the same author participates. In the beginning the paper talks about "coarse discrete methods" and may give the impression that it contains some continuous time simulation method. It really doesn't -see the description of their algorithm in the Appendix, page 30.

Also [this MPhil dissertation](https://people.maths.ox.ac.uk/witte/JHWitte_DPhilThesis_13_March_2012.pdf.), which, being a graduate thesis, describes the steps in some detail, and with a wealth of referenced literature.

> **Comments:**
> **username_0**: Both Ben's notes and his paper don't really mention how to simulate them, only how to solve them - and that's not my problem..  

---

### Answer by username_3 on 2015-09-10. Score: 1

Since Ben's notes were already mentioned I'm not quite sure why you don't iterate over time using a forward approximation as he suggests
\begin{align}
\dot k(t) :\approx \frac{k(t+1) - k(t)}{\Delta t} \Leftrightarrow k(t+1) \approx \Delta t \dot k(t) + k(t)
\end{align}

which gives
\begin{align}
k(1) \approx \Delta t (z-c(0)) + k(0)
\end{align}

> **Comments:**
> **username_4**: You can also look at the approximations made with finite difference equation. It is a good method do discretize a continous time model.  

---

### Answer by username_5 on 2014-11-19. Score: 0

Once you have solved the equation $\rho V(k) = \max\_c g(c) + V'(k)(z - c)$
to find the function $V(k)$ you should be able to solve for what is called the policy function. The function $V(k)$ is the value function. Knowing $V(k)$ should then give you the
information that you need to solve for the policy, say $c(k,t)$. This will then tell you how $k$ will evolve over time.

> **Comments:**
> **username_0**: I know $c(k,t)$, my problem is how to understand it. How *exactly* do I get $k(1)$, given that I know the policy function and $k(0)$?  

---

