title: Infinity-MM: Scaling Multimodal Performance with Large-Scale and
    High-Quality Instruction Data
summary: Vision-Language Models (VLMs) have recently made significant progress, but
  the limited scale and quality of open-source instruction data hinder their
  performance compared to closed-source models. In this work, we address this
  limitation by introducing Infinity-MM, a large-scale multimodal instruction
  dataset with 40 million samples, enhanced through rigorous quality filtering
  and deduplication. We also propose a synthetic instruction generation method
  based on open-source VLMs, using detailed image annotations and diverse
  question generation. Using this data, we trained a 2-billion-parameter VLM,
  Aquila-VL-2B, achieving state-of-the-art (SOTA) performance for models of
  similar scale. This demonstrates that expanding instruction data and generating
  synthetic data can significantly improve the performance of open-source models.
paper_authors._id: 671afdd1ffcdeeb655cbe21b
paper_authors.name: Shuhao Gu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-28T15:19:19.039Z
paper_authors._id: 642e72cec1b0f8e4e76af16d
paper_authors.avatarUrl: /avatars/f900811d3c22a114c67283b646949f86.svg
paper_authors.fullname: shuhao gu
paper_authors.type: user
paper_authors.user: gsh33
paper_authors._id: 671afdd1ffcdeeb655cbe21c
paper_authors.name: Jialing Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-15T09:42:08.927Z
paper_authors._id: 65473b574486732720548dae
paper_authors.avatarUrl: /avatars/180ff1bd6a58b87d702c411c2df6b663.svg
paper_authors.fullname: Jialing Zhang
paper_authors.type: user
paper_authors.user: JialingZhang
paper_authors._id: 671afdd1ffcdeeb655cbe21d
paper_authors.name: Siyuan Zhou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-30T09:54:50.318Z
paper_authors._id: 663e2aef881fe6a203c681e8
paper_authors.avatarUrl: /avatars/4bcdd7db684cf98e6881f1ee29b2e16c.svg
paper_authors.fullname: Siyuan Zhou
paper_authors.type: user
paper_authors.user: Ssyuan9876
paper_authors._id: 671afdd1ffcdeeb655cbe21e
paper_authors.name: Kevin Yu
paper_authors._id: 671afdd1ffcdeeb655cbe21f
paper_authors.name: Zhaohu Xing
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-28T15:20:13.818Z
paper_authors._id: 632c2100ea6e62428ab201e9
paper_authors.avatarUrl: /avatars/5bd355e095af93261928198e3d6d5696.svg
paper_authors.fullname: Xingzhaohu
paper_authors.type: user
paper_authors.user: xingzhaohu
paper_authors._id: 671afdd1ffcdeeb655cbe220
paper_authors.name: Liangdong Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-25T09:30:05.708Z
paper_authors._id: 63a11ce02fabbbb899a01d58
paper_authors.avatarUrl: /avatars/ee3d4088b6d32b2c18b8be91913e90dd.svg
paper_authors.fullname: ldwang
paper_authors.type: user
paper_authors.user: ldwang
paper_authors._id: 671afdd1ffcdeeb655cbe221
paper_authors.name: Zhou Cao
paper_authors._id: 671afdd1ffcdeeb655cbe222
paper_authors.name: Jintao Jia
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-28T15:20:22.639Z
paper_authors._id: 6695dc41b91490da0249797e
paper_authors.avatarUrl: /avatars/59df94f34fb4967e39b8237aac669546.svg
paper_authors.fullname: Jintao Jia
paper_authors.type: user
paper_authors.user: jjtnb
paper_authors._id: 671afdd1ffcdeeb655cbe223
paper_authors.name: Zhuoyi Zhang
paper_authors._id: 671afdd1ffcdeeb655cbe224
paper_authors.name: Yixuan Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-28T16:55:00.224Z
paper_authors._id: 66a0a5721bb371390c70f5a5
paper_authors.avatarUrl: /avatars/9403480ef1277ac789ad2d375f398113.svg
paper_authors.fullname: Grace Wang
paper_authors.type: user
paper_authors.user: Gracecc
paper_authors._id: 671afdd1ffcdeeb655cbe225
paper_authors.name: Zhenchong Hu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-28T15:21:14.573Z
paper_authors._id: 669600f8cebc15b84aff0337
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/N755P9kTEM4Vu_UUY_lAU.jpeg
paper_authors.fullname: 胡振崇
paper_authors.type: user
paper_authors.user: zhenchonghu
paper_authors._id: 671afdd1ffcdeeb655cbe226
paper_authors.name: Bo-Wen Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-28T11:04:53.176Z
paper_authors._id: 6335113375bed9932474315e
paper_authors.avatarUrl: /avatars/aa1837a3a07514b89c48de888934a7b2.svg
paper_authors.fullname: bowenzhang
paper_authors.type: user
paper_authors.user: bowen92
paper_authors._id: 671afdd1ffcdeeb655cbe227
paper_authors.name: Jijie Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-28T15:21:27.092Z
paper_authors._id: 65e00e64092e556a9a7dd429
paper_authors.avatarUrl: /avatars/57959174ef1b0a9dbb14e53d4aea58a8.svg
paper_authors.fullname: lijijie
paper_authors.type: user
paper_authors.user: Avoczh
paper_authors._id: 671afdd1ffcdeeb655cbe228
paper_authors.name: Dong Liang
paper_authors._id: 671afdd1ffcdeeb655cbe229
paper_authors.name: Yingli Zhao
paper_authors._id: 671afdd1ffcdeeb655cbe22a
paper_authors.name: Yulong Ao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-28T15:22:10.950Z
paper_authors._id: 64f3eb4fd41e83d74a0173ec
paper_authors.avatarUrl: /avatars/461f804bb5a7c5b31f862cfa27e52de1.svg
paper_authors.fullname: Yulong Ao
paper_authors.type: user
paper_authors.user: aoyulong
paper_authors._id: 671afdd1ffcdeeb655cbe22b
paper_authors.name: Yaoqi Liu
paper_authors._id: 671afdd1ffcdeeb655cbe22c
paper_authors.name: Fangxiang Feng
paper_authors._id: 671afdd1ffcdeeb655cbe22d
paper_authors.name: Guang Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-28T11:22:42.703Z
paper_authors._id: 632c234f42c386ebd2710434
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/632c234f42c386ebd2710434/HyWRWi063S69JTy_IMjoe.jpeg
paper_authors.fullname: Guang Liu
paper_authors.type: user
paper_authors.user: ZacLiu
paper_title: Infinity-MM: Scaling Multimodal Performance with Large-Scale and
    High-Quality Instruction Data
paper_summary: Vision-Language Models (VLMs) have recently made significant progress, but
  the limited scale and quality of open-source instruction data hinder their
  performance compared to closed-source models. In this work, we address this
  limitation by introducing Infinity-MM, a large-scale multimodal instruction
  dataset with 40 million samples, enhanced through rigorous quality filtering
  and deduplication. We also propose a synthetic instruction generation method
  based on open-source VLMs, using detailed image annotations and diverse
  question generation. Using this data, we trained a 2-billion-parameter VLM,
  Aquila-VL-2B, achieving state-of-the-art (SOTA) performance for models of
  similar scale. This demonstrates that expanding instruction data and generating
  synthetic data can significantly improve the performance of open-source models.
paper_ai_summary: Infinity-MM, a large-scale multimodal instruction dataset, enhances open-source VLMs through rigorous filtering and synthetic data generation, achieving state-of-the-art performance in models of similar size.
paper_ai_keywords: Vision-Language Models multimodal instruction dataset quality filtering deduplication synthetic instruction generation Aquila-VL-2B state-of-the-art performance