title: The Synergy between Data and Multi-Modal Large Language Models: A Survey
    from Co-Development Perspective
summary: The rapid development of large language models (LLMs) has been witnessed in
  recent years. Based on the powerful LLMs, multi-modal LLMs (MLLMs) extend the
  modality from text to a broader spectrum of domains, attracting widespread
  attention due to the broader range of application scenarios. As LLMs and MLLMs
  rely on vast amounts of model parameters and data to achieve emergent
  capabilities, the importance of data is receiving increasingly widespread
  attention and recognition. Tracing and analyzing recent data-oriented works for
  MLLMs, we find that the development of models and data is not two separate
  paths but rather interconnected. On the one hand, vaster and higher-quality
  data contribute to better performance of MLLMs, on the other hand, MLLMs can
  facilitate the development of data. The co-development of multi-modal data and
  MLLMs requires a clear view of 1) at which development stage of MLLMs can
  specific data-centric approaches be employed to enhance which capabilities, and
  2) by utilizing which capabilities and acting as which roles can models
  contribute to multi-modal data. To promote the data-model co-development for
  MLLM community, we systematically review existing works related to MLLMs from
  the data-model co-development perspective. A regularly maintained project
  associated with this survey is accessible at
  https://github.com/modelscope/data-juicer/blob/main/docs/awesome_llm_data.md.
paper_authors._id: 66908e4644464c66a819e74e
paper_authors.name: Zhen Qin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-12T07:40:14.493Z
paper_authors._id: 649014b91d71e55664838d2d
paper_authors.avatarUrl: /avatars/f0e0f2830c5cb7428cbbc9634d95c34b.svg
paper_authors.fullname: Zhen Qin
paper_authors.type: user
paper_authors.user: zhenqincn
paper_authors._id: 66908e4644464c66a819e74f
paper_authors.name: Daoyuan Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-12T10:09:23.541Z
paper_authors._id: 6530ca70c1b8f265a431fc42
paper_authors.avatarUrl: /avatars/7b1ec5b75398c8434ccd43cdce7fee53.svg
paper_authors.fullname: Daoyuan Chen
paper_authors.type: user
paper_authors.user: yxdyc
paper_authors._id: 66908e4644464c66a819e750
paper_authors.name: Wenhao Zhang
paper_authors._id: 66908e4644464c66a819e751
paper_authors.name: Liuyi Yao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-12T10:09:39.519Z
paper_authors._id: 649148b63c7b190918fc9214
paper_authors.avatarUrl: /avatars/1dc51c948cd8e4ab716a8b199ef04387.svg
paper_authors.fullname: Liuyi Yao
paper_authors.type: user
paper_authors.user: Briayao
paper_authors._id: 66908e4644464c66a819e752
paper_authors.name: Yilun Huang
paper_authors._id: 66908e4644464c66a819e753
paper_authors.name: Bolin Ding
paper_authors._id: 66908e4644464c66a819e754
paper_authors.name: Yaliang Li
paper_authors._id: 66908e4644464c66a819e755
paper_authors.name: Shuiguang Deng
paper_title: The Synergy between Data and Multi-Modal Large Language Models: A Survey
    from Co-Development Perspective
paper_summary: The rapid development of large language models (LLMs) has been witnessed in
  recent years. Based on the powerful LLMs, multi-modal LLMs (MLLMs) extend the
  modality from text to a broader spectrum of domains, attracting widespread
  attention due to the broader range of application scenarios. As LLMs and MLLMs
  rely on vast amounts of model parameters and data to achieve emergent
  capabilities, the importance of data is receiving increasingly widespread
  attention and recognition. Tracing and analyzing recent data-oriented works for
  MLLMs, we find that the development of models and data is not two separate
  paths but rather interconnected. On the one hand, vaster and higher-quality
  data contribute to better performance of MLLMs, on the other hand, MLLMs can
  facilitate the development of data. The co-development of multi-modal data and
  MLLMs requires a clear view of 1) at which development stage of MLLMs can
  specific data-centric approaches be employed to enhance which capabilities, and
  2) by utilizing which capabilities and acting as which roles can models
  contribute to multi-modal data. To promote the data-model co-development for
  MLLM community, we systematically review existing works related to MLLMs from
  the data-model co-development perspective. A regularly maintained project
  associated with this survey is accessible at
  https://github.com/modelscope/data-juicer/blob/main/docs/awesome_llm_data.md.
paper_ai_summary: A review of the co-development between multi-modal language models and data highlights the interdependence between model advancements and data quality.
paper_ai_keywords: large language models multi-modal language models model parameters vaster and higher-quality data co-development data-centric approaches multi-modal data