DiscoVLA: Discrepancy Reduction in Vision, Language, and Alignment for
  Parameter-Efficient Video-Text Retrieval
The parameter-efficient adaptation of the image-text pretraining model CLIP
for video-text retrieval is a prominent area of research. While CLIP is focused
on image-level vision-language matching, video-text retrieval demands
comprehensive understanding at the video level. Three key discrepancies emerge
in the transfer from image-level to video-level: vision, language, and
alignment. However, existing methods mainly focus on vision while neglecting
language and alignment. In this paper, we propose Discrepancy Reduction in
Vision, Language, and Alignment (DiscoVLA), which simultaneously mitigates all
three discrepancies. Specifically, we introduce Image-Video Features Fusion to
integrate image-level and video-level features, effectively tackling both
vision and language discrepancies. Additionally, we generate pseudo image
captions to learn fine-grained image-level alignment. To mitigate alignment
discrepancies, we propose Image-to-Video Alignment Distillation, which
leverages image-level alignment knowledge to enhance video-level alignment.
Extensive experiments demonstrate the superiority of our DiscoVLA. In
particular, on MSRVTT with CLIP (ViT-B/16), DiscoVLA outperforms previous
methods by 1.5% in R@1, reaching a final score of 50.5% R@1. The code is
available at https://github.com/LunarShen/DsicoVLA.
6848ec1542e4f9106973f2ac
Leqi Shen
claimed_verified
2025-06-11T08:35:05.097Z
6364b81b3e248b1e28a68b26
https://cdn-avatars.huggingface.co/v1/production/uploads/6364b81b3e248b1e28a68b26/9pd6zfH3HGx1gQkYuL3pR.png
LeqiShen
user
LeqiShen
6848ec1542e4f9106973f2ad
Guoqiang Gong
6848ec1542e4f9106973f2ae
Tianxiang Hao
claimed_verified
2025-07-02T08:55:23.481Z
67fc7ba101fde81f68836d0a
/avatars/482657bba0dc0226df2098b89f915cd9.svg
Tianxiang Hao
user
beyondhtx
6848ec1542e4f9106973f2af
Tao He
6848ec1542e4f9106973f2b0
Yifeng Zhang
6848ec1542e4f9106973f2b1
Pengzhang Liu
6848ec1542e4f9106973f2b2
Sicheng Zhao
6848ec1542e4f9106973f2b3
Jungong Han
6848ec1542e4f9106973f2b4
Guiguang Ding
DiscoVLA: Discrepancy Reduction in Vision, Language, and Alignment for
  Parameter-Efficient Video-Text Retrieval
The parameter-efficient adaptation of the image-text pretraining model CLIP
for video-text retrieval is a prominent area of research. While CLIP is focused
on image-level vision-language matching, video-text retrieval demands
comprehensive understanding at the video level. Three key discrepancies emerge
in the transfer from image-level to video-level: vision, language, and
alignment. However, existing methods mainly focus on vision while neglecting
language and alignment. In this paper, we propose Discrepancy Reduction in
Vision, Language, and Alignment (DiscoVLA), which simultaneously mitigates all
three discrepancies. Specifically, we introduce Image-Video Features Fusion to
integrate image-level and video-level features, effectively tackling both
vision and language discrepancies. Additionally, we generate pseudo image
captions to learn fine-grained image-level alignment. To mitigate alignment
discrepancies, we propose Image-to-Video Alignment Distillation, which
leverages image-level alignment knowledge to enhance video-level alignment.
Extensive experiments demonstrate the superiority of our DiscoVLA. In
particular, on MSRVTT with CLIP (ViT-B/16), DiscoVLA outperforms previous
methods by 1.5% in R@1, reaching a final score of 50.5% R@1. The code is
available at https://github.com/LunarShen/DsicoVLA.
The paper proposes DiscoVLA to improve video-text retrieval using CLIP by addressing vision, language, and alignment discrepancies, achieving superior performance.
parameter-efficient adaptation image-text pretraining model CLIP video-text retrieval vision language alignment Image-Video Features Fusion pseudo image captions Image-to-Video Alignment Distillation MSRVTT R@1