HOIDiffusion: Generating Realistic 3D Hand-Object Interaction Data
<|reference_start|>HOIDiffusion: Generating Realistic 3D Hand-Object Interaction Data: 3D hand-object interaction data is scarce due to the hardware constraints in scaling up the data collection process. In this paper, we propose HOIDiffusion for generating realistic and diverse 3D hand-object interaction data. Our model is a conditional diffusion model that takes both the 3D hand-object geometric structure and text description as inputs for image synthesis. This offers a more controllable and realistic synthesis as we can specify the structure and style inputs in a disentangled manner. HOIDiffusion is trained by leveraging a diffusion model pre-trained on large-scale natural images and a few 3D human demonstrations. Beyond controllable image synthesis, we adopt the generated 3D data for learning 6D object pose estimation and show its effectiveness in improving perception systems. Project page: https://mq-zhang1.github.io/HOIDiffusion<|reference_end|>
@article{zhang2024hoidiffusion:,
    title={HOIDiffusion: Generating Realistic 3D Hand-Object Interaction Data},
    author={Mengqi Zhang, Yang Fu, Zheng Ding, Sifei Liu, Zhuowen Tu, Xiaolong
  Wang},
    journal={arXiv preprint arXiv:2403.12011},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.12011},
    primaryClass={cs.CV}
}