Safe Deep Model-Based Reinforcement Learning with Lyapunov Functions
<|reference_start|>Safe Deep Model-Based Reinforcement Learning with Lyapunov Functions: Model-based Reinforcement Learning (MBRL) has shown many desirable properties for intelligent control tasks. However, satisfying safety and stability constraints during training and rollout remains an open question. We propose a new Model-based RL framework to enable efficient policy learning with unknown dynamics based on learning model predictive control (LMPC) framework with mathematically provable guarantees of stability. We introduce and explore a novel method for adding safety constraints for model-based RL during training and policy learning. The new stability-augmented framework consists of a neural-network-based learner that learns to construct a Lyapunov function, and a model-based RL agent to consistently complete the tasks while satisfying user-specified constraints given only sub-optimal demonstrations and sparse-cost feedback. We demonstrate the capability of the proposed framework through simulated experiments.<|reference_end|>
@article{zhang2024safe,
    title={Safe Deep Model-Based Reinforcement Learning with Lyapunov Functions},
    author={Harry Zhang},
    journal={arXiv preprint arXiv:2405.16184},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.16184},
    primaryClass={eess.SY cs.AI cs.LG cs.SY}
}