Vision-and-Language Navigation via Causal Learning
<|reference_start|>Vision-and-Language Navigation via Causal Learning: In the pursuit of robust and generalizable environment perception and language understanding, the ubiquitous challenge of dataset bias continues to plague vision-and-language navigation (VLN) agents, hindering their performance in unseen environments. This paper introduces the generalized cross-modal causal transformer (GOAT), a pioneering solution rooted in the paradigm of causal inference. By delving into both observable and unobservable confounders within vision, language, and history, we propose the back-door and front-door adjustment causal learning (BACL and FACL) modules to promote unbiased learning by comprehensively mitigating potential spurious correlations. Additionally, to capture global confounder features, we propose a cross-modal feature pooling (CFP) module supervised by contrastive learning, which is also shown to be effective in improving cross-modal representations during pre-training. Extensive experiments across multiple VLN datasets (R2R, REVERIE, RxR, and SOON) underscore the superiority of our proposed method over previous state-of-the-art approaches. Code is available at https://github.com/CrystalSixone/VLN-GOAT.<|reference_end|>
@article{wang2024vision-and-language,
    title={Vision-and-Language Navigation via Causal Learning},
    author={Liuyi Wang, Zongtao He, Ronghao Dang, Mengjiao Shen, Chengju Liu,
  Qijun Chen},
    journal={arXiv preprint arXiv:2404.10241},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.10241},
    primaryClass={cs.CV cs.AI}
}