ConstrainedZero: Chance-Constrained POMDP Planning using Learned Probabilistic Failure Surrogates and Adaptive Safety Constraints
<|reference_start|>ConstrainedZero: Chance-Constrained POMDP Planning using Learned Probabilistic Failure Surrogates and Adaptive Safety Constraints: To plan safely in uncertain environments, agents must balance utility with safety constraints. Safe planning problems can be modeled as a chance-constrained partially observable Markov decision process (CC-POMDP) and solutions often use expensive rollouts or heuristics to estimate the optimal value and action-selection policy. This work introduces the ConstrainedZero policy iteration algorithm that solves CC-POMDPs in belief space by learning neural network approximations of the optimal value and policy with an additional network head that estimates the failure probability given a belief. This failure probability guides safe action selection during online Monte Carlo tree search (MCTS). To avoid overemphasizing search based on the failure estimates, we introduce $\Delta$-MCTS, which uses adaptive conformal inference to update the failure threshold during planning. The approach is tested on a safety-critical POMDP benchmark, an aircraft collision avoidance system, and the sustainability problem of safe CO$_2$ storage. Results show that by separating safety constraints from the objective we can achieve a target level of safety without optimizing the balance between rewards and costs.<|reference_end|>
@article{moss2024constrainedzero:,
    title={ConstrainedZero: Chance-Constrained POMDP Planning using Learned
  Probabilistic Failure Surrogates and Adaptive Safety Constraints},
    author={Robert J. Moss, Arec Jamgochian, Johannes Fischer, Anthony Corso, and
  Mykel J. Kochenderfer},
    journal={arXiv preprint arXiv:2405.00644},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.00644},
    primaryClass={cs.AI}
}