Batch Active Learning of Reward Functions from Human Preferences
<|reference_start|>Batch Active Learning of Reward Functions from Human Preferences: Data generation and labeling are often expensive in robot learning. Preference-based learning is a concept that enables reliable labeling by querying users with preference questions. Active querying methods are commonly employed in preference-based learning to generate more informative data at the expense of parallelization and computation time. In this paper, we develop a set of novel algorithms, batch active preference-based learning methods, that enable efficient learning of reward functions using as few data samples as possible while still having short query generation times and also retaining parallelizability. We introduce a method based on determinantal point processes (DPP) for active batch generation and several heuristic-based alternatives. Finally, we present our experimental results for a variety of robotics tasks in simulation. Our results suggest that our batch active learning algorithm requires only a few queries that are computed in a short amount of time. We showcase one of our algorithms in a study to learn human users' preferences.<|reference_end|>
@article{bıyık2024batch,
    title={Batch Active Learning of Reward Functions from Human Preferences},
    author={Erdem B{i}y{i}k, Nima Anari, Dorsa Sadigh},
    journal={arXiv preprint arXiv:2402.15757},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.15757},
    primaryClass={cs.LG cs.AI cs.RO stat.ML}
}