GQE: Generalized Query Expansion for Enhanced Text-Video Retrieval
<|reference_start|>GQE: Generalized Query Expansion for Enhanced Text-Video Retrieval: In the rapidly expanding domain of web video content, the task of text-video retrieval has become increasingly critical, bridging the semantic gap between textual queries and video data. This paper introduces a novel data-centric approach, Generalized Query Expansion (GQE), to address the inherent information imbalance between text and video, enhancing the effectiveness of text-video retrieval systems. Unlike traditional model-centric methods that focus on designing intricate cross-modal interaction mechanisms, GQE aims to expand the text queries associated with videos both during training and testing phases. By adaptively segmenting videos into short clips and employing zero-shot captioning, GQE enriches the training dataset with comprehensive scene descriptions, effectively bridging the data imbalance gap. Furthermore, during retrieval, GQE utilizes Large Language Models (LLM) to generate a diverse set of queries and a query selection module to filter these queries based on relevance and diversity, thus optimizing retrieval performance while reducing computational overhead. Our contributions include a detailed examination of the information imbalance challenge, a novel approach to query expansion in video-text datasets, and the introduction of a query selection strategy that enhances retrieval accuracy without increasing computational costs. GQE achieves state-of-the-art performance on several benchmarks, including MSR-VTT, MSVD, LSMDC, and VATEX, demonstrating the effectiveness of addressing text-video retrieval from a data-centric perspective.<|reference_end|>
@article{bai2024gqe:,
    title={GQE: Generalized Query Expansion for Enhanced Text-Video Retrieval},
    author={Zechen Bai, Tianjun Xiao, Tong He, Pichao Wang, Zheng Zhang, Thomas
  Brox, Mike Zheng Shou},
    journal={arXiv preprint arXiv:2408.07249},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.07249},
    primaryClass={cs.CV cs.IR}
}