A Pontryagin Perspective on Reinforcement Learning
<|reference_start|>A Pontryagin Perspective on Reinforcement Learning: Reinforcement learning has traditionally focused on learning state-dependent policies to solve optimal control problems in a closed-loop fashion. In this work, we introduce the paradigm of open-loop reinforcement learning where a fixed action sequence is learned instead. We present three new algorithms: one robust model-based method and two sample-efficient model-free methods. Rather than basing our algorithms on Bellman's equation from dynamic programming, our work builds on Pontryagin's principle from the theory of open-loop optimal control. We provide convergence guarantees and evaluate all methods empirically on a pendulum swing-up task, as well as on two high-dimensional MuJoCo tasks, demonstrating remarkable performance compared to existing baselines.<|reference_end|>
@article{eberhard2024a,
    title={A Pontryagin Perspective on Reinforcement Learning},
    author={Onno Eberhard, Claire Vernade, Michael Muehlebach},
    journal={arXiv preprint arXiv:2405.18100},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.18100},
    primaryClass={cs.LG math.OC}
}