Reducing Human-Robot Goal State Divergence with Environment Design
<|reference_start|>Reducing Human-Robot Goal State Divergence with Environment Design: One of the most difficult challenges in creating successful human-AI collaborations is aligning a robot's behavior with a human user's expectations. When this fails to occur, a robot may misinterpret their specified goals, prompting it to perform actions with unanticipated, potentially dangerous side effects. To avoid this, we propose a new metric we call Goal State Divergence $\mathcal{(GSD)}$, which represents the difference between a robot's final goal state and the one a human user expected. In cases where $\mathcal{GSD}$ cannot be directly calculated, we show how it can be approximated using maximal and minimal bounds. We then input the $\mathcal{GSD}$ value into our novel human-robot goal alignment (HRGA) design problem, which identifies a minimal set of environment modifications that can prevent mismatches like this. To show the effectiveness of $\mathcal{GSD}$ for reducing differences between human-robot goal states, we empirically evaluate our approach on several standard benchmarks.<|reference_end|>
@article{sikes2024reducing,
    title={Reducing Human-Robot Goal State Divergence with Environment Design},
    author={Kelsey Sikes, Sarah Keren, Sarath Sreedharan},
    journal={arXiv preprint arXiv:2404.15184},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.15184},
    primaryClass={cs.AI}
}