Diffusion-Driven Self-Supervised Learning for Shape Reconstruction and Pose Estimation
<|reference_start|>Diffusion-Driven Self-Supervised Learning for Shape Reconstruction and Pose Estimation: Fully-supervised category-level pose estimation aims to determine the 6-DoF poses of unseen instances from known categories, requiring expensive mannual labeling costs. Recently, various self-supervised category-level pose estimation methods have been proposed to reduce the requirement of the annotated datasets. However, most methods rely on synthetic data or 3D CAD model for self-supervised training, and they are typically limited to addressing single-object pose problems without considering multi-objective tasks or shape reconstruction. To overcome these challenges and limitations, we introduce a diffusion-driven self-supervised network for multi-object shape reconstruction and categorical pose estimation, only leveraging the shape priors. Specifically, to capture the SE(3)-equivariant pose features and 3D scale-invariant shape information, we present a Prior-Aware Pyramid 3D Point Transformer in our network. This module adopts a point convolutional layer with radial-kernels for pose-aware learning and a 3D scale-invariant graph convolution layer for object-level shape representation, respectively. Furthermore, we introduce a pretrain-to-refine self-supervised training paradigm to train our network. It enables proposed network to capture the associations between shape priors and observations, addressing the challenge of intra-class shape variations by utilising the diffusion mechanism. Extensive experiments conducted on four public datasets and a self-built dataset demonstrate that our method significantly outperforms state-of-the-art self-supervised category-level baselines and even surpasses some fully-supervised instance-level and category-level methods.<|reference_end|>
@article{sun2024diffusion-driven,
    title={Diffusion-Driven Self-Supervised Learning for Shape Reconstruction and
  Pose Estimation},
    author={Jingtao Sun, Yaonan Wang, Mingtao Feng, Chao Ding, Mike Zheng Shou and
  Ajmal Saeed Mian},
    journal={arXiv preprint arXiv:2403.12728},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.12728},
    primaryClass={cs.CV}
}