Structure-Aware Residual-Center Representation for Self-Supervised Open-Set 3D Cross-Modal Retrieval
<|reference_start|>Structure-Aware Residual-Center Representation for Self-Supervised Open-Set 3D Cross-Modal Retrieval: Existing methods of 3D cross-modal retrieval heavily lean on category distribution priors within the training set, which diminishes their efficacy when tasked with unseen categories under open-set environments. To tackle this problem, we propose the Structure-Aware Residual-Center Representation (SRCR) framework for self-supervised open-set 3D cross-modal retrieval. To address the center deviation due to category distribution differences, we utilize the Residual-Center Embedding (RCE) for each object by nested auto-encoders, rather than directly mapping them to the modality or category centers. Besides, we perform the Hierarchical Structure Learning (HSL) approach to leverage the high-order correlations among objects for generalization, by constructing a heterogeneous hypergraph structure based on hierarchical inter-modality, intra-object, and implicit-category correlations. Extensive experiments and ablation studies on four benchmarks demonstrate the superiority of our proposed framework compared to state-of-the-art methods.<|reference_end|>
@article{xu2024structure-aware,
    title={Structure-Aware Residual-Center Representation for Self-Supervised
  Open-Set 3D Cross-Modal Retrieval},
    author={Yang Xu, Yifan Feng, and Yu Jiang},
    journal={arXiv preprint arXiv:2407.15376},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.15376},
    primaryClass={cs.MM}
}