Uncertainty-aware Active Learning of NeRF-based Object Models for Robot Manipulators using Visual and Re-orientation Actions
<|reference_start|>Uncertainty-aware Active Learning of NeRF-based Object Models for Robot Manipulators using Visual and Re-orientation Actions: Manipulating unseen objects is challenging without a 3D representation, as objects generally have occluded surfaces. This requires physical interaction with objects to build their internal representations. This paper presents an approach that enables a robot to rapidly learn the complete 3D model of a given object for manipulation in unfamiliar orientations. We use an ensemble of partially constructed NeRF models to quantify model uncertainty to determine the next action (a visual or re-orientation action) by optimizing informativeness and feasibility. Further, our approach determines when and how to grasp and re-orient an object given its partial NeRF model and re-estimates the object pose to rectify misalignments introduced during the interaction. Experiments with a simulated Franka Emika Robot Manipulator operating in a tabletop environment with benchmark objects demonstrate an improvement of (i) 14% in visual reconstruction quality (PSNR), (ii) 20% in the geometric/depth reconstruction of the object surface (F-score) and (iii) 71% in the task success rate of manipulating objects a-priori unseen orientations/stable configurations in the scene; over current methods. The project page can be found here: https://actnerf.github.io.<|reference_end|>
@article{dasgupta2024uncertainty-aware,
    title={Uncertainty-aware Active Learning of NeRF-based Object Models for Robot
  Manipulators using Visual and Re-orientation Actions},
    author={Saptarshi Dasgupta, Akshat Gupta, Shreshth Tuli, Rohan Paul},
    journal={arXiv preprint arXiv:2404.01812},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.01812},
    primaryClass={cs.RO cs.AI}
}