title: MegaPairs: Massive Data Synthesis For Universal Multimodal Retrieval
summary: Despite the rapidly growing demand for multimodal retrieval, progress in this
  field remains severely constrained by a lack of training data. In this paper,
  we introduce MegaPairs, a novel data synthesis method that leverages vision
  language models (VLMs) and open-domain images, together with a massive
  synthetic dataset generated from this method. Our empirical analysis shows that
  MegaPairs generates high-quality data, enabling the multimodal retriever to
  significantly outperform the baseline model trained on 70times more data
  from existing datasets. Moreover, since MegaPairs solely relies on general
  image corpora and open-source VLMs, it can be easily scaled up, enabling
  continuous improvements in retrieval performance. In this stage, we produced
  more than 26 million training instances and trained several models of varying
  sizes using this data. These new models achieve state-of-the-art zero-shot
  performance across 4 popular composed image retrieval (CIR) benchmarks and the
  highest overall performance on the 36 datasets provided by MMEB. They also
  demonstrate notable performance improvements with additional downstream
  fine-tuning. Our produced dataset, well-trained models, and data synthesis
  pipeline will be made publicly available to facilitate the future development
  of this field.
paper_authors._id: 6764da4e0044fe72b24b009e
paper_authors.name: Junjie Zhou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-20T08:32:59.134Z
paper_authors._id: 6564a2ceedae9c33b7654a1f
paper_authors.avatarUrl: /avatars/42f09356a1282896573ccb44830cd327.svg
paper_authors.fullname: JUNJIE ZHOU
paper_authors.type: user
paper_authors.user: JUNJIE99
paper_authors._id: 6764da4e0044fe72b24b009f
paper_authors.name: Zheng Liu
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-12-20T02:45:36.277Z
paper_authors._id: 64a38c590111d5ff6c3d5f2b
paper_authors.avatarUrl: /avatars/ef13dc7ce243819bc0da9b04e778b432.svg
paper_authors.fullname: zhengliu
paper_authors.type: user
paper_authors.user: zl101
paper_authors._id: 6764da4e0044fe72b24b00a0
paper_authors.name: Ze Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-21T15:19:54.180Z
paper_authors._id: 66164f6245336ca774679611
paper_authors.avatarUrl: /avatars/9baf0ab475bc8d5997abda9ffe8cfa28.svg
paper_authors.fullname: Ze Liu
paper_authors.type: user
paper_authors.user: marsh123
paper_authors._id: 6764da4e0044fe72b24b00a1
paper_authors.name: Shitao Xiao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-20T09:17:36.576Z
paper_authors._id: 62612679bbcbd1c34f1638af
paper_authors.avatarUrl: /avatars/c0675d05a52192ee14e9ab1633353956.svg
paper_authors.fullname: Xiao
paper_authors.type: user
paper_authors.user: Shitao
paper_authors._id: 6764da4e0044fe72b24b00a2
paper_authors.name: Yueze Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-20T09:17:27.209Z
paper_authors._id: 6458b59c7a7e192202df8fa0
paper_authors.avatarUrl: /avatars/33ee716477e5686da8723d01e199cd27.svg
paper_authors.fullname: Yueze Wang
paper_authors.type: user
paper_authors.user: yzwang
paper_authors._id: 6764da4e0044fe72b24b00a3
paper_authors.name: Bo Zhao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-20T09:17:20.202Z
paper_authors._id: 64acafac1aee69ece03af05a
paper_authors.avatarUrl: /avatars/6380777852546dbd432fed5544d5db0b.svg
paper_authors.fullname: Bo Zhao
paper_authors.type: user
paper_authors.user: Z-LIRA
paper_authors._id: 6764da4e0044fe72b24b00a4
paper_authors.name: Chen Jason Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-20T09:17:04.629Z
paper_authors._id: 6465943fe27766e8921bf4b3
paper_authors.avatarUrl: /avatars/86cc7f24c2348c634711807c3261961d.svg
paper_authors.fullname: chenjason
paper_authors.type: user
paper_authors.user: chenjason
paper_authors._id: 6764da4e0044fe72b24b00a5
paper_authors.name: Defu Lian
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-20T09:16:55.115Z
paper_authors._id: 66ed026076a8038cb4ae6053
paper_authors.avatarUrl: /avatars/99b6527da6b66c6b5df3fc8261587322.svg
paper_authors.fullname: Defu Lian
paper_authors.type: user
paper_authors.user: dove-ustc
paper_authors._id: 6764da4e0044fe72b24b00a6
paper_authors.name: Yongping Xiong
paper_title: MegaPairs: Massive Data Synthesis For Universal Multimodal Retrieval
paper_summary: Despite the rapidly growing demand for multimodal retrieval, progress in this
  field remains severely constrained by a lack of training data. In this paper,
  we introduce MegaPairs, a novel data synthesis method that leverages vision
  language models (VLMs) and open-domain images, together with a massive
  synthetic dataset generated from this method. Our empirical analysis shows that
  MegaPairs generates high-quality data, enabling the multimodal retriever to
  significantly outperform the baseline model trained on 70times more data
  from existing datasets. Moreover, since MegaPairs solely relies on general
  image corpora and open-source VLMs, it can be easily scaled up, enabling
  continuous improvements in retrieval performance. In this stage, we produced
  more than 26 million training instances and trained several models of varying
  sizes using this data. These new models achieve state-of-the-art zero-shot
  performance across 4 popular composed image retrieval (CIR) benchmarks and the
  highest overall performance on the 36 datasets provided by MMEB. They also
  demonstrate notable performance improvements with additional downstream
  fine-tuning. Our produced dataset, well-trained models, and data synthesis
  pipeline will be made publicly available to facilitate the future development
  of this field.
paper_ai_summary: MegaPairs, a data synthesis method using vision language models and open-domain images, generates high-quality multimodal retrieval data, outperforming larger existing datasets and achieving state-of-the-art zero-shot performance.
paper_ai_keywords: vision language models multimodal retrieval data synthesis MegaPairs composed image retrieval MMEB