MERIT: Multilingual Semantic Retrieval with Interleaved Multi-Condition
  Query
Semantic retrieval is crucial for modern applications yet remains
underexplored in current research. Existing datasets are limited to single
languages, single images, or singular retrieval conditions, often failing to
fully exploit the expressive capacity of visual information as evidenced by
maintained performance when images are replaced with captions. However,
practical retrieval scenarios frequently involve interleaved multi-condition
queries with multiple images. Hence, this paper introduces MERIT, the first
multilingual dataset for interleaved multi-condition semantic retrieval,
comprising 320,000 queries with 135,000 products in 5 languages, covering 7
distinct product categories. Extensive experiments on MERIT identify existing
models's limitation: focusing solely on global semantic information while
neglecting specific conditional elements in queries. Consequently, we propose
Coral, a novel fine-tuning framework that adapts pre-trained MLLMs by
integrating embedding reconstruction to preserve fine-grained conditional
elements and contrastive learning to extract comprehensive global semantics.
Experiments demonstrate that Coral achieves a 45.9% performance improvement
over conventional approaches on MERIT, with strong generalization capabilities
validated across 8 established retrieval benchmarks. Collectively, our
contributions - a novel dataset, identification of critical limitations in
existing approaches, and an innovative fine-tuning framework - establish a
foundation for future research in interleaved multi-condition semantic
retrieval.
684015f08bd5bff99191dff4
Wei Chow
claimed_verified
2025-06-04T09:56:45.423Z
644b71ddb2e7823a76abcf91
https://cdn-avatars.huggingface.co/v1/production/uploads/644b71ddb2e7823a76abcf91/JPF7Eqeq2jx8i79nQ962K.jpeg
zhou wei
user
WeiChow
684015f08bd5bff99191dff5
Yuan Gao
684015f08bd5bff99191dff6
Linfeng Li
claimed_verified
2025-06-05T07:27:55.480Z
67c952eb648c10bac4b0f5a2
https://cdn-avatars.huggingface.co/v1/production/uploads/67c952eb648c10bac4b0f5a2/Da2XIF3Pvt3T2tLFVg6GZ.jpeg
Yolox
user
linfengup
684015f08bd5bff99191dff7
Xian Wang
684015f08bd5bff99191dff8
Qi Xu
684015f08bd5bff99191dff9
Hang Song
684015f08bd5bff99191dffa
Lingdong Kong
684015f08bd5bff99191dffb
Ran Zhou
684015f08bd5bff99191dffc
Yi Zeng
684015f08bd5bff99191dffd
Yidong Cai
684015f08bd5bff99191dffe
Botian Jiang
684015f08bd5bff99191dfff
Shilin Xu
684015f08bd5bff99191e000
Jiajun Zhang
684015f08bd5bff99191e001
Minghui Qiu
684015f08bd5bff99191e002
Xiangtai Li
claimed_verified
2025-06-11T08:36:57.875Z
63958b4414513eaf9029ebf1
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/U1g5H071pWRswGAG9UTpo.png
Xiangtai Li
user
LXT
684015f08bd5bff99191e003
Tianshu Yang
684015f08bd5bff99191e004
Siliang Tang
684015f08bd5bff99191e005
Juncheng Li
MERIT: Multilingual Semantic Retrieval with Interleaved Multi-Condition
  Query
Semantic retrieval is crucial for modern applications yet remains
underexplored in current research. Existing datasets are limited to single
languages, single images, or singular retrieval conditions, often failing to
fully exploit the expressive capacity of visual information as evidenced by
maintained performance when images are replaced with captions. However,
practical retrieval scenarios frequently involve interleaved multi-condition
queries with multiple images. Hence, this paper introduces MERIT, the first
multilingual dataset for interleaved multi-condition semantic retrieval,
comprising 320,000 queries with 135,000 products in 5 languages, covering 7
distinct product categories. Extensive experiments on MERIT identify existing
models's limitation: focusing solely on global semantic information while
neglecting specific conditional elements in queries. Consequently, we propose
Coral, a novel fine-tuning framework that adapts pre-trained MLLMs by
integrating embedding reconstruction to preserve fine-grained conditional
elements and contrastive learning to extract comprehensive global semantics.
Experiments demonstrate that Coral achieves a 45.9% performance improvement
over conventional approaches on MERIT, with strong generalization capabilities
validated across 8 established retrieval benchmarks. Collectively, our
contributions - a novel dataset, identification of critical limitations in
existing approaches, and an innovative fine-tuning framework - establish a
foundation for future research in interleaved multi-condition semantic
retrieval.
MERIT, a multilingual dataset for interleaved multi-condition semantic retrieval, highlights limitations in existing models and introduces Coral, a fine-tuning framework integrating embedding reconstruction and contrastive learning to improve retrieval performance.
MULTILINGUAL dataset interleaved multi-condition semantic retrieval pre-trained MLLMs embedding reconstruction contrastive learning retrieval benchmarks