Multi-Task Domain Adaptation for Language Grounding with 3D Objects
<|reference_start|>Multi-Task Domain Adaptation for Language Grounding with 3D Objects: The existing works on object-level language grounding with 3D objects mostly focus on improving performance by utilizing the off-the-shelf pre-trained models to capture features, such as viewpoint selection or geometric priors. However, they have failed to consider exploring the cross-modal representation of language-vision alignment in the cross-domain field. To answer this problem, we propose a novel method called Domain Adaptation for Language Grounding (DA4LG) with 3D objects. Specifically, the proposed DA4LG consists of a visual adapter module with multi-task learning to realize vision-language alignment by comprehensive multimodal feature representation. Experimental results demonstrate that DA4LG competitively performs across visual and non-visual language descriptions, independent of the completeness of observation. DA4LG achieves state-of-the-art performance in the single-view setting and multi-view setting with the accuracy of 83.8% and 86.8% respectively in the language grounding benchmark SNARE. The simulation experiments show the well-practical and generalized performance of DA4LG compared to the existing methods. Our project is available at https://sites.google.com/view/da4lg.<|reference_end|>
@article{sun2024multi-task,
    title={Multi-Task Domain Adaptation for Language Grounding with 3D Objects},
    author={Penglei Sun, Yaoxian Song, Xinglin Pan, Peijie Dong, Xiaofei Yang,
  Qiang Wang, Zhixu Li, Tiefeng Li, and Xiaowen Chu},
    journal={arXiv preprint arXiv:2407.02846},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.02846},
    primaryClass={cs.CV}
}