2DP-2MRC: 2-Dimensional Pointer-based Machine Reading Comprehension Method for Multimodal Moment Retrieval
<|reference_start|>2DP-2MRC: 2-Dimensional Pointer-based Machine Reading Comprehension Method for Multimodal Moment Retrieval: Moment retrieval aims to locate the most relevant moment in an untrimmed video based on a given natural language query. Existing solutions can be roughly categorized into moment-based and clip-based methods. The former often involves heavy computations, while the latter, due to overlooking coarse-grained information, typically underperforms compared to moment-based models. Hence, this paper proposes a novel 2-Dimensional Pointer-based Machine Reading Comprehension for Moment Retrieval Choice (2DP-2MRC) model to address the issue of imprecise localization in clip-based methods while maintaining lower computational complexity than moment-based methods. Specifically, we introduce an AV-Encoder to capture coarse-grained information at moment and video levels. Additionally, a 2D pointer encoder module is introduced to further enhance boundary detection for target moment. Extensive experiments on the HiREST dataset demonstrate that 2DP-2MRC significantly outperforms existing baseline models.<|reference_end|>
@article{he20242dp-2mrc:,
    title={2DP-2MRC: 2-Dimensional Pointer-based Machine Reading Comprehension
  Method for Multimodal Moment Retrieval},
    author={Jiajun He, Tomoki Toda},
    journal={arXiv preprint arXiv:2406.06201},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.06201},
    primaryClass={cs.CV cs.AI}
}