Grasp, See and Place: Efficient Unknown Object Rearrangement with Policy Structure Prior
<|reference_start|>Grasp, See and Place: Efficient Unknown Object Rearrangement with Policy Structure Prior: We focus on the task of unknown object rearrangement, where a robot is supposed to re-configure the objects into a desired goal configuration specified by an RGB-D image. Recent works explore unknown object rearrangement systems by incorporating learning-based perception modules. However, they are sensitive to perception error, and pay less attention to task-level performance. In this paper, we aim to develop an effective system for unknown object rearrangement amidst perception noise. We theoretically reveal the noisy perception impacts grasp and place in a decoupled way, and show such a decoupled structure is valuable to improve task optimality. We propose GSP, a dual-loop system with the decoupled structure as prior. For the inner loop, we learn a see policy for self-confident in-hand object matching. For the outer loop, we learn a grasp policy aware of object matching and grasp capability guided by task-level rewards. We leverage the foundation model CLIP for object matching, policy learning and self-termination. A series of experiments indicate that GSP can conduct unknown object rearrangement with higher completion rates and fewer steps.<|reference_end|>
@article{xu2024grasp,,
    title={Grasp, See and Place: Efficient Unknown Object Rearrangement with Policy
  Structure Prior},
    author={Kechun Xu, Zhongxiang Zhou, Jun Wu, Haojian Lu, Rong Xiong, Yue Wang},
    journal={arXiv preprint arXiv:2402.15402},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.15402},
    primaryClass={cs.RO cs.LG}
}