title: MERIT: Multilingual Semantic Retrieval with Interleaved Multi-Condition
    Query
summary: Semantic retrieval is crucial for modern applications yet remains
  underexplored in current research. Existing datasets are limited to single
  languages, single images, or singular retrieval conditions, often failing to
  fully exploit the expressive capacity of visual information as evidenced by
  maintained performance when images are replaced with captions. However,
  practical retrieval scenarios frequently involve interleaved multi-condition
  queries with multiple images. Hence, this paper introduces MERIT, the first
  multilingual dataset for interleaved multi-condition semantic retrieval,
  comprising 320,000 queries with 135,000 products in 5 languages, covering 7
  distinct product categories. Extensive experiments on MERIT identify existing
  models's limitation: focusing solely on global semantic information while
  neglecting specific conditional elements in queries. Consequently, we propose
  Coral, a novel fine-tuning framework that adapts pre-trained MLLMs by
  integrating embedding reconstruction to preserve fine-grained conditional
  elements and contrastive learning to extract comprehensive global semantics.
  Experiments demonstrate that Coral achieves a 45.9% performance improvement
  over conventional approaches on MERIT, with strong generalization capabilities
  validated across 8 established retrieval benchmarks. Collectively, our
  contributions - a novel dataset, identification of critical limitations in
  existing approaches, and an innovative fine-tuning framework - establish a
  foundation for future research in interleaved multi-condition semantic
  retrieval.
paper_authors._id: 684015f08bd5bff99191dff4
paper_authors.name: Wei Chow
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-04T09:56:45.423Z
paper_authors._id: 644b71ddb2e7823a76abcf91
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/644b71ddb2e7823a76abcf91/JPF7Eqeq2jx8i79nQ962K.jpeg
paper_authors.fullname: zhou wei
paper_authors.type: user
paper_authors.user: WeiChow
paper_authors._id: 684015f08bd5bff99191dff5
paper_authors.name: Yuan Gao
paper_authors._id: 684015f08bd5bff99191dff6
paper_authors.name: Linfeng Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-05T07:27:55.480Z
paper_authors._id: 67c952eb648c10bac4b0f5a2
paper_authors.avatarUrl: /avatars/accf4f82d5b54824e89b0c2ce70163ce.svg
paper_authors.fullname: Yolox
paper_authors.type: user
paper_authors.user: linfengup
paper_authors._id: 684015f08bd5bff99191dff7
paper_authors.name: Xian Wang
paper_authors._id: 684015f08bd5bff99191dff8
paper_authors.name: Qi Xu
paper_authors._id: 684015f08bd5bff99191dff9
paper_authors.name: Hang Song
paper_authors._id: 684015f08bd5bff99191dffa
paper_authors.name: Lingdong Kong
paper_authors._id: 684015f08bd5bff99191dffb
paper_authors.name: Ran Zhou
paper_authors._id: 684015f08bd5bff99191dffc
paper_authors.name: Yi Zeng
paper_authors._id: 684015f08bd5bff99191dffd
paper_authors.name: Yidong Cai
paper_authors._id: 684015f08bd5bff99191dffe
paper_authors.name: Botian Jiang
paper_authors._id: 684015f08bd5bff99191dfff
paper_authors.name: Shilin Xu
paper_authors._id: 684015f08bd5bff99191e000
paper_authors.name: Jiajun Zhang
paper_authors._id: 684015f08bd5bff99191e001
paper_authors.name: Minghui Qiu
paper_authors._id: 684015f08bd5bff99191e002
paper_authors.name: Xiangtai Li
paper_authors._id: 684015f08bd5bff99191e003
paper_authors.name: Tianshu Yang
paper_authors._id: 684015f08bd5bff99191e004
paper_authors.name: Siliang Tang
paper_authors._id: 684015f08bd5bff99191e005
paper_authors.name: Juncheng Li
paper_title: MERIT: Multilingual Semantic Retrieval with Interleaved Multi-Condition
    Query
paper_summary: Semantic retrieval is crucial for modern applications yet remains
  underexplored in current research. Existing datasets are limited to single
  languages, single images, or singular retrieval conditions, often failing to
  fully exploit the expressive capacity of visual information as evidenced by
  maintained performance when images are replaced with captions. However,
  practical retrieval scenarios frequently involve interleaved multi-condition
  queries with multiple images. Hence, this paper introduces MERIT, the first
  multilingual dataset for interleaved multi-condition semantic retrieval,
  comprising 320,000 queries with 135,000 products in 5 languages, covering 7
  distinct product categories. Extensive experiments on MERIT identify existing
  models's limitation: focusing solely on global semantic information while
  neglecting specific conditional elements in queries. Consequently, we propose
  Coral, a novel fine-tuning framework that adapts pre-trained MLLMs by
  integrating embedding reconstruction to preserve fine-grained conditional
  elements and contrastive learning to extract comprehensive global semantics.
  Experiments demonstrate that Coral achieves a 45.9% performance improvement
  over conventional approaches on MERIT, with strong generalization capabilities
  validated across 8 established retrieval benchmarks. Collectively, our
  contributions - a novel dataset, identification of critical limitations in
  existing approaches, and an innovative fine-tuning framework - establish a
  foundation for future research in interleaved multi-condition semantic
  retrieval.
paper_ai_summary: MERIT, a multilingual dataset for interleaved multi-condition semantic retrieval, highlights limitations in existing models and introduces Coral, a fine-tuning framework integrating embedding reconstruction and contrastive learning to improve retrieval performance.
paper_ai_keywords: MULTILINGUAL dataset interleaved multi-condition semantic retrieval pre-trained MLLMs embedding reconstruction contrastive learning retrieval benchmarks