On Bellman equations for continuous-time policy evaluation I: discretization and approximation
<|reference_start|>On Bellman equations for continuous-time policy evaluation I: discretization and approximation: We study the problem of computing the value function from a discretely-observed trajectory of a continuous-time diffusion process. We develop a new class of algorithms based on easily implementable numerical schemes that are compatible with discrete-time reinforcement learning (RL) with function approximation. We establish high-order numerical accuracy as well as the approximation error guarantees for the proposed approach. In contrast to discrete-time RL problems where the approximation factor depends on the effective horizon, we obtain a bounded approximation factor using the underlying elliptic structures, even if the effective horizon diverges to infinity.<|reference_end|>
@article{mou2024on,
    title={On Bellman equations for continuous-time policy evaluation I:
  discretization and approximation},
    author={Wenlong Mou, Yuhua Zhu},
    journal={arXiv preprint arXiv:2407.05966},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.05966},
    primaryClass={cs.LG cs.NA math.NA math.OC math.PR}
}