Entropy-regularized Point-based Value Iteration
<|reference_start|>Entropy-regularized Point-based Value Iteration: Model-based planners for partially observable problems must accommodate both model uncertainty during planning and goal uncertainty during objective inference. However, model-based planners may be brittle under these types of uncertainty because they rely on an exact model and tend to commit to a single optimal behavior. Inspired by results in the model-free setting, we propose an entropy-regularized model-based planner for partially observable problems. Entropy regularization promotes policy robustness for planning and objective inference by encouraging policies to be no more committed to a single action than necessary. We evaluate the robustness and objective inference performance of entropy-regularized policies in three problem domains. Our results show that entropy-regularized policies outperform non-entropy-regularized baselines in terms of higher expected returns under modeling errors and higher accuracy during objective inference.<|reference_end|>
@article{delecki2024entropy-regularized,
    title={Entropy-regularized Point-based Value Iteration},
    author={Harrison Delecki, Marcell Vazquez-Chanlatte, Esen Yel, Kyle Wray,
  Tomer Arnon, Stefan Witwicki, Mykel J. Kochenderfer},
    journal={arXiv preprint arXiv:2402.09388},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.09388},
    primaryClass={cs.AI}
}