<p>Consider a routine continuous time optimization problem:<br />
<span class="math-container">$
V(t,a_{t}) := 
\max \int_{\tau=t}^{\tau = T} 
e^{-\rho (\tau -t)} u(c_{\tau})d\tau
$</span> <span class="math-container">$\text{  s.t. }$</span><br />
<span class="math-container">$\dot{a}_{t} = y + ra_{t} - c_{t}$</span>,<br />
<span class="math-container">$a_{0} \text{ given, } a_{T}=0.$</span><br />
Assume y &amp; r are constants and <span class="math-container">$u(c)=\frac{c^{1-\gamma}}{1-\gamma}$</span>.</p>
<p><strong>Optimal control approach</strong>:<br />
<span class="math-container">$H := u(c_{t}) + \lambda (y + ra_{t} - c_{t})$</span><br />
Boils down to a BVP (two ODEs w/ unkown <span class="math-container">$c_{t}, a_{t}$</span>):<br />
<span class="math-container">$\left[\begin{array}{l}
\dot{c}=\left(\frac{r-\rho}{\gamma} \right) c
\\
\dot{a}_{t} = y + ra_{t} - c_{t}
\\
a(0)=a_0
\\ 
a_{T}=0
\end{array} \right]$</span>.<br />
This can be plugged into a <a href="https://github.com/SciML/DifferentialEquations.jl" rel="nofollow noreferrer">DiffEq</a> solver bc we have two first-order ODEs w/ two boundary conditions.</p>
<p><strong>HJB approach</strong>:<br />
<span class="math-container">$\rho V(t,a_{t}) = \max_{c} \left\{u(c_{t}) + V_{a}(t,a_{t})\times \left(y + ra_{t} - c_{t} \right) + V_{t}(t,a_{t}) \right\}$</span><br />
FOC: <span class="math-container">$c(t,a_{t})=u'^{-1}(V_{a}(t,a_{t})) = (V_{a}(t,a_{t}))^{-\frac{1}{\gamma}}$</span><br />
This boils down to a PDE w/ unknown function <span class="math-container">$V(t,a_{t})$</span>:<br />
<span class="math-container">$\left[\begin{array}{l}
\rho V(t,a_{t}) = u(c_{t}) + V_{a}(t,a_{t})\times \left(y + ra_{t} - c_{t} \right) + V_{t}(t,a_{t}) 
\\
c(t,a_{t}) = (V_{a}(t,a_{t}))^{-\frac{1}{\gamma}}
\end{array} \right]$</span><br />
To solve this first-order PDE we need one boundary condition for <span class="math-container">$V(t,a_{t})$</span>.<br />
<strong>Issue</strong>: we have boundary conditions for <span class="math-container">$a_{t}$</span> <span class="math-container">$(a(0)=a_0, a_{T}=0)$</span> but not for <span class="math-container">$V(t,a_{t})$</span>.<br />
<strong>Question</strong>: How do we get the boundary conditions for HJBs in general?</p>
<p><strong>Partial answer</strong> (case 1): finite-horizon deterministic problems<br />
<span class="math-container">$V(T,a_{T}) = \psi(a_{T}) = 0$</span>   (no bequest here by assumption)
<span class="math-container">$\left[\begin{array}{l}
\rho V(t,a_{t}) = u(c_{t}) + V_{a}(t,a_{t})\times \left(y + ra_{t} - c_{t} \right) + V_{t}(t,a_{t}) 
\\
c(t,a_{t}) = (V_{a}(t,a_{t}))^{-\frac{1}{\gamma}}
\\
V(T,a) = 0 \text{ , } \forall a
\end{array} \right]$</span></p>
<p>Generically, many problems in economics can be written:<br />
<span class="math-container">$V(t,x_{t}) := 
\max \int_{\tau=t}^{\tau = T} e^{-\rho (\tau -t)}u(\tau,x_{\tau},c_{\tau})d\tau +
\Omega(T,x_{T})$</span>  <span class="math-container">$\text{  s.t. }$</span><br />
<span class="math-container">$\dot{x}_{t} = g(t,x_{t},c_{t})$</span>,<br />
<span class="math-container">$x_{0} \text{ given, } x_{T}=0.$</span></p>
<p>The generic optimal control problem is straightforward, w/ boundary conditions <span class="math-container">$x_{0}, x_{T}=0$</span>.</p>
<p>Generic deterministic HJB:<br />
<span class="math-container">$\left[\begin{array}{l}
\rho V(t,x_{t}) = \max_{c} \left\{u(t,x_{t},c_{t}) 
+ V_{x}(t,x_{t}) \dot{x} + V_{t}(t,x_{t}) \right\} 
\\
V(T,x_{T}) = \Omega(T,x)  \text{ , } \forall x
\end{array} \right]$</span></p>
<p><strong>Case 2</strong>: finite horizon stochastic problems (w/ Ito processes)<br />
<span class="math-container">$V(t,x_{t}) := 
\max \int_{\tau=t}^{\tau = T} e^{-\rho (\tau -t)} u(\tau,x_{\tau},c_{\tau})d\tau + \Omega(T,x_{T})$</span>  <span class="math-container">$\text{  s.t. }$</span><br />
<span class="math-container">$dx=\mu_{t}dt + \sigma_{t} dz_{t}$</span>, where <span class="math-container">$\mu_{t} = g(t,x_{t},c_{t})$</span>,<br />
<span class="math-container">$x_{0} \text{ given, } x_{T}=0.$</span><br />
<span class="math-container">$\rho V(t,x_{t}) 
= 
\max_{c} \left\{ u(t,x_{t},c_{t}) 
+ E_{t}\frac{dV}{dt}\right\} 
$</span>.<br />
Ito's Lemma: if <span class="math-container">$X \sim dX=\mu_{t}dt + \sigma_{t} dZ_{t}$</span> <span class="math-container">$\Rightarrow$</span>
<span class="math-container">$E_{t}\frac{dV(t,x_{t})}{dt} =
V_{t}(t,x_{t})  + \mu_{t}V_{x}(t,x_{t}) 
+ \frac{\sigma_{t}^2}{2} V_{xx}(t,x_{t})
$</span><br />
<span class="math-container">$\left[\begin{array}{l}
\rho V(t,x_{t}) = 
\max_{c} \left\{ u(t, x_{t},c_{t}) 
+ V_{t}(t,x_{t})  + \mu_{t}V_{x}(t,x_{t}) 
+ \frac{\sigma_{t}^2}{2} V_{xx}(t,x_{t})
\right\} 
\\
V(T,x_{T}) = \Omega(T,x)   \text{ , } \forall x
\end{array} \right]$</span><br />
<strong>Question 2</strong>: a generic HJB for a stochastic problem (w/ Ito risk) is a second-order PDE and we need two boundary conditions.<br />
What is the second boundary condition for <span class="math-container">$V(t,x_{t})$</span>?<br />
My guess:<br />
<span class="math-container">$\left[\begin{array}{l}
\rho V(t,x_{t}) = 
\max_{c} \left\{ u(x_{t},c_{t},t) 
+ V_{t}(t,x_{t})  + \mu_{t}V_{x}(t,x_{t}) 
+ \frac{\sigma_{t}^2}{2} V_{xx}(t,x_{t})
\right\} 
\\
V(T,x_{T}) = \Omega(T,x)    \text{ , } \forall x
\\
V_{x}(T,x_{T}) = \Omega_{x}(T,x)    \text{ , } \forall x
\end{array} \right]$</span></p>
<p>The two boundary conditions for the finite-horizon pdes above are usually called:<br />
Value matching: <span class="math-container">$V(T,x_{T}) = \Omega(T,x_{T}) $</span><br />
Smooth pasting: <span class="math-container">$V_{x}(T,x_{T}) = \Omega_{x}(T,x_{T}) $</span><br />
In the original example <span class="math-container">$\Omega(T,x_{T}) =0$</span>.</p>
<p>Remaining questions: what are the value matching &amp; smooth pasting conditions for generic infinite horizon problems?<br />
I've seen:<br />
<span class="math-container">$\lim_{t \to \infty} E[ e^{-\rho t} V(x_{t}) ]=0$</span><br />
<span class="math-container">$\lim_{t \to \infty} E[ e^{-\rho t} V_{x}(x_{t}) ]=0$</span></p>
<p><strong>Case 3</strong>:
What is the boundary condition for an infinite horizon deterministic HJB?</p>
<p><strong>Case 4</strong>:
What is the boundary condition for an infinite horizon stochastic HJB?</p>
<p><strong>Case 5</strong>:
Problems with borrowing constraints etc studied by Moll and Gomez et al?</p>
<p><strong>Lesson 1</strong>: PDEs are ALOT more <a href="https://math.stackexchange.com/questions/663640/why-cant-we-say-that-all-pdes-of-a-specified-order-require-a-fixed-number-of-bo?rq=1">complicated</a> than ODEs.<br />
An nth order ODE needs n boundary conditions.<br />
An nth order hyperbolic PDE (wave eqn) needs n boundary conditions.<br />
An nth order elliptic equation needs n/2 boundary conditions.<br />
Is there a general rule for the HJB PDEs economists usually solve?</p>
<p><strong>Lesson 2</strong>: HJB PDEs are usually solved w/ viscosity methods.</p>
<p>See Moll, <a href="https://benjaminmoll.com/wp-content/uploads/2020/06/lifecycle.pdf" rel="nofollow noreferrer">Lifecycle (finite horizon)</a>.<br />
See Moll, <a href="https://benjaminmoll.com/wp-content/uploads/2019/07/Lecture3_2149.pdf" rel="nofollow noreferrer">Lecture 3, Slide 11</a>.<br />
<a href="https://benjaminmoll.com/wp-content/uploads/2019/07/HACT.pdf" rel="nofollow noreferrer">Achdou, Han, Lasry, Lions, Moll (2020)</a> is the most seminal paper in this literature, but doesn't really answer this question.<br />
This is challenging b/c it's hard to plug a pde into a pde-solver w/o the proper boundary conditions.</p>

CC BY-SA 4.0
2020-11-07T04:26:10.327Z
The terminology "value matching" and "smoothing pasting" are not being used correctly here. They occur in *free boundary* problems---e.g. exercise of American options or real options. There is no "boundary condition" at infinity for HJB equations. Rather HJB conditions are only necessary. Sufficiency is given by transversality-type conditions at infinity. In stochastic control, such sufficiency results are called *verification theorems*. One needs to first solve the HJB, then apply verification (second step commonly not done in economics).
Michael
CC BY-SA 4.0
2020-11-07T05:04:27.397Z
Whatever the terminology, those conditions give the correct solution in the first two cases. 
See Laibson's [lecture notes](https://projects.iq.harvard.edu/files/econ2010c/files/lectures_9-10_2010c_2014.pdf).
Albert Zevelev
CC BY-SA 4.0
2020-11-08T02:05:23.840Z
Evidently you're misreading the terminology from the linked notes.
Michael
<optimization><dynamic-programming><optimal-control><continuous-time><bellman-equations>
What are the boundary value conditions for generic HJBs in economics?
Albert Zevelev
# What are the boundary value conditions for generic HJBs in economics?

(Asked by: username_0 on 2020-11-05)

Consider a routine continuous time optimization problem:  
$
V(t,a\_{t}) :=
\max \int\_{\tau=t}^{\tau = T}
e^{-\rho (\tau -t)} u(c\_{\tau})d\tau
$ $\text{ s.t. }$  
$\dot{a}\_{t} = y + ra\_{t} - c\_{t}$,  
$a\_{0} \text{ given, } a\_{T}=0.$  
Assume y & r are constants and $u(c)=\frac{c^{1-\gamma}}{1-\gamma}$.

**Optimal control approach**:  
$H := u(c\_{t}) + \lambda (y + ra\_{t} - c\_{t})$  
Boils down to a BVP (two ODEs w/ unkown $c\_{t}, a\_{t}$):  
$\left[\begin{array}{l}
\dot{c}=\left(\frac{r-\rho}{\gamma} \right) c
\\
\dot{a}\_{t} = y + ra\_{t} - c\_{t}
\\
a(0)=a\_0
\\
a\_{T}=0
\end{array} \right]$.  
This can be plugged into a [DiffEq](https://github.com/SciML/DifferentialEquations.jl) solver bc we have two first-order ODEs w/ two boundary conditions.

**HJB approach**:  
$\rho V(t,a\_{t}) = \max\_{c} \left\{u(c\_{t}) + V\_{a}(t,a\_{t})\times \left(y + ra\_{t} - c\_{t} \right) + V\_{t}(t,a\_{t}) \right\}$  
FOC: $c(t,a\_{t})=u'^{-1}(V\_{a}(t,a\_{t})) = (V\_{a}(t,a\_{t}))^{-\frac{1}{\gamma}}$  
This boils down to a PDE w/ unknown function $V(t,a\_{t})$:  
$\left[\begin{array}{l}
\rho V(t,a\_{t}) = u(c\_{t}) + V\_{a}(t,a\_{t})\times \left(y + ra\_{t} - c\_{t} \right) + V\_{t}(t,a\_{t})
\\
c(t,a\_{t}) = (V\_{a}(t,a\_{t}))^{-\frac{1}{\gamma}}
\end{array} \right]$  
To solve this first-order PDE we need one boundary condition for $V(t,a\_{t})$.  
**Issue**: we have boundary conditions for $a\_{t}$ $(a(0)=a\_0, a\_{T}=0)$ but not for $V(t,a\_{t})$.  
**Question**: How do we get the boundary conditions for HJBs in general?

**Partial answer** (case 1): finite-horizon deterministic problems  
$V(T,a\_{T}) = \psi(a\_{T}) = 0$ (no bequest here by assumption)
$\left[\begin{array}{l}
\rho V(t,a\_{t}) = u(c\_{t}) + V\_{a}(t,a\_{t})\times \left(y + ra\_{t} - c\_{t} \right) + V\_{t}(t,a\_{t})
\\
c(t,a\_{t}) = (V\_{a}(t,a\_{t}))^{-\frac{1}{\gamma}}
\\
V(T,a) = 0 \text{ , } \forall a
\end{array} \right]$

Generically, many problems in economics can be written:  
$V(t,x\_{t}) :=
\max \int\_{\tau=t}^{\tau = T} e^{-\rho (\tau -t)}u(\tau,x\_{\tau},c\_{\tau})d\tau +
\Omega(T,x\_{T})$ $\text{ s.t. }$  
$\dot{x}\_{t} = g(t,x\_{t},c\_{t})$,  
$x\_{0} \text{ given, } x\_{T}=0.$

The generic optimal control problem is straightforward, w/ boundary conditions $x\_{0}, x\_{T}=0$.

Generic deterministic HJB:  
$\left[\begin{array}{l}
\rho V(t,x\_{t}) = \max\_{c} \left\{u(t,x\_{t},c\_{t})
+ V\_{x}(t,x\_{t}) \dot{x} + V\_{t}(t,x\_{t}) \right\}
\\
V(T,x\_{T}) = \Omega(T,x) \text{ , } \forall x
\end{array} \right]$

**Case 2**: finite horizon stochastic problems (w/ Ito processes)  
$V(t,x\_{t}) :=
\max \int\_{\tau=t}^{\tau = T} e^{-\rho (\tau -t)} u(\tau,x\_{\tau},c\_{\tau})d\tau + \Omega(T,x\_{T})$ $\text{ s.t. }$  
$dx=\mu\_{t}dt + \sigma\_{t} dz\_{t}$, where $\mu\_{t} = g(t,x\_{t},c\_{t})$,  
$x\_{0} \text{ given, } x\_{T}=0.$  
$\rho V(t,x\_{t})
=
\max\_{c} \left\{ u(t,x\_{t},c\_{t})
+ E\_{t}\frac{dV}{dt}\right\}
$.  
Ito's Lemma: if $X \sim dX=\mu\_{t}dt + \sigma\_{t} dZ\_{t}$ $\Rightarrow$
$E\_{t}\frac{dV(t,x\_{t})}{dt} =
V\_{t}(t,x\_{t}) + \mu\_{t}V\_{x}(t,x\_{t})
+ \frac{\sigma\_{t}^2}{2} V\_{xx}(t,x\_{t})
$  
$\left[\begin{array}{l}
\rho V(t,x\_{t}) =
\max\_{c} \left\{ u(t, x\_{t},c\_{t})
+ V\_{t}(t,x\_{t}) + \mu\_{t}V\_{x}(t,x\_{t})
+ \frac{\sigma\_{t}^2}{2} V\_{xx}(t,x\_{t})
\right\}
\\
V(T,x\_{T}) = \Omega(T,x) \text{ , } \forall x
\end{array} \right]$  
**Question 2**: a generic HJB for a stochastic problem (w/ Ito risk) is a second-order PDE and we need two boundary conditions.  
What is the second boundary condition for $V(t,x\_{t})$?  
My guess:  
$\left[\begin{array}{l}
\rho V(t,x\_{t}) =
\max\_{c} \left\{ u(x\_{t},c\_{t},t)
+ V\_{t}(t,x\_{t}) + \mu\_{t}V\_{x}(t,x\_{t})
+ \frac{\sigma\_{t}^2}{2} V\_{xx}(t,x\_{t})
\right\}
\\
V(T,x\_{T}) = \Omega(T,x) \text{ , } \forall x
\\
V\_{x}(T,x\_{T}) = \Omega\_{x}(T,x) \text{ , } \forall x
\end{array} \right]$

The two boundary conditions for the finite-horizon pdes above are usually called:  
Value matching: $V(T,x\_{T}) = \Omega(T,x\_{T}) $  
Smooth pasting: $V\_{x}(T,x\_{T}) = \Omega\_{x}(T,x\_{T}) $  
In the original example $\Omega(T,x\_{T}) =0$.

Remaining questions: what are the value matching & smooth pasting conditions for generic infinite horizon problems?  
I've seen:  
$\lim\_{t \to \infty} E[ e^{-\rho t} V(x\_{t}) ]=0$  
$\lim\_{t \to \infty} E[ e^{-\rho t} V\_{x}(x\_{t}) ]=0$

**Case 3**:
What is the boundary condition for an infinite horizon deterministic HJB?

**Case 4**:
What is the boundary condition for an infinite horizon stochastic HJB?

**Case 5**:
Problems with borrowing constraints etc studied by Moll and Gomez et al?

**Lesson 1**: PDEs are ALOT more [complicated](https://math.stackexchange.com/questions/663640/why-cant-we-say-that-all-pdes-of-a-specified-order-require-a-fixed-number-of-bo?rq=1) than ODEs.  
An nth order ODE needs n boundary conditions.  
An nth order hyperbolic PDE (wave eqn) needs n boundary conditions.  
An nth order elliptic equation needs n/2 boundary conditions.  
Is there a general rule for the HJB PDEs economists usually solve?

**Lesson 2**: HJB PDEs are usually solved w/ viscosity methods.

See Moll, [Lifecycle (finite horizon)](https://benjaminmoll.com/wp-content/uploads/2020/06/lifecycle.pdf).  
See Moll, [Lecture 3, Slide 11](https://benjaminmoll.com/wp-content/uploads/2019/07/Lecture3_2149.pdf).  
[Achdou, Han, Lasry, Lions, Moll (2020)](https://benjaminmoll.com/wp-content/uploads/2019/07/HACT.pdf) is the most seminal paper in this literature, but doesn't really answer this question.  
This is challenging b/c it's hard to plug a pde into a pde-solver w/o the proper boundary conditions.

**Tags:** optimization, dynamic-programming, optimal-control, continuous-time, bellman-equations

> **Comments:**
> **username_1**: The terminology "value matching" and "smoothing pasting" are not being used correctly here. They occur in *free boundary* problems---e.g. exercise of American options or real options. There is no "boundary condition" at infinity for HJB equations. Rather HJB conditions are only necessary. Sufficiency is given by transversality-type conditions at infinity. In stochastic control, such sufficiency results are called *verification theorems*. One needs to first solve the HJB, then apply verification (second step commonly not done in economics).  
> **username_0**: Whatever the terminology, those conditions give the correct solution in the first two cases.  See Laibson's [lecture notes](https://projects.iq.harvard.edu/files/econ2010c/files/lectures_9-10_2010c_2014.pdf).  
> **username_1**: Evidently you're misreading the terminology from the linked notes.  

---

## 0 answers

