EAGLE: Efficient Adaptive Geometry-based Learning in Cross-view Understanding
<|reference_start|>EAGLE: Efficient Adaptive Geometry-based Learning in Cross-view Understanding: Unsupervised Domain Adaptation has been an efficient approach to transferring the semantic segmentation model across data distributions. Meanwhile, the recent Open-vocabulary Semantic Scene understanding based on large-scale vision language models is effective in open-set settings because it can learn diverse concepts and categories. However, these prior methods fail to generalize across different camera views due to the lack of cross-view geometric modeling. At present, there are limited studies analyzing cross-view learning. To address this problem, we introduce a novel Unsupervised Cross-view Adaptation Learning approach to modeling the geometric structural change across views in Semantic Scene Understanding. First, we introduce a novel Cross-view Geometric Constraint on Unpaired Data to model structural changes in images and segmentation masks across cameras. Second, we present a new Geodesic Flow-based Correlation Metric to efficiently measure the geometric structural changes across camera views. Third, we introduce a novel view-condition prompting mechanism to enhance the view-information modeling of the open-vocabulary segmentation network in cross-view adaptation learning. The experiments on different cross-view adaptation benchmarks have shown the effectiveness of our approach in cross-view modeling, demonstrating that we achieve State-of-the-Art (SOTA) performance compared to prior unsupervised domain adaptation and open-vocabulary semantic segmentation methods.<|reference_end|>
@article{truong2024eagle:,
    title={EAGLE: Efficient Adaptive Geometry-based Learning in Cross-view
  Understanding},
    author={Thanh-Dat Truong, Utsav Prabhu, Dongyi Wang, Bhiksha Raj, Susan Gauch,
  Jeyamkondan Subbiah, Khoa Luu},
    journal={arXiv preprint arXiv:2406.01429},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.01429},
    primaryClass={cs.CV}
}