title: Data-Juicer Sandbox: A Comprehensive Suite for Multimodal Data-Model
    Co-development
summary: The emergence of large-scale multi-modal generative models has drastically
  advanced artificial intelligence, introducing unprecedented levels of
  performance and functionality. However, optimizing these models remains
  challenging due to historically isolated paths of model-centric and
  data-centric developments, leading to suboptimal outcomes and inefficient
  resource utilization. In response, we present a novel sandbox suite tailored
  for integrated data-model co-development. This sandbox provides a comprehensive
  experimental platform, enabling rapid iteration and insight-driven refinement
  of both data and models. Our proposed "Probe-Analyze-Refine" workflow,
  validated through applications on state-of-the-art LLaVA-like and DiT based
  models, yields significant performance boosts, such as topping the VBench
  leaderboard. We also uncover fruitful insights gleaned from exhaustive
  benchmarks, shedding light on the critical interplay between data quality,
  diversity, and model behavior. With the hope of fostering deeper understanding
  and future progress in multi-modal data and generative modeling, our codes,
  datasets, and models are maintained and accessible at
  https://github.com/modelscope/data-juicer/blob/main/docs/Sandbox.md.
paper_authors._id: 669739baa7fd582ae7420a37
paper_authors.name: Daoyuan Chen
paper_authors._id: 669739baa7fd582ae7420a38
paper_authors.name: Haibin Wang
paper_authors._id: 669739baa7fd582ae7420a39
paper_authors.name: Yilun Huang
paper_authors._id: 669739baa7fd582ae7420a3a
paper_authors.name: Ce Ge
paper_authors._id: 669739baa7fd582ae7420a3b
paper_authors.name: Yaliang Li
paper_authors._id: 669739baa7fd582ae7420a3c
paper_authors.name: Bolin Ding
paper_authors._id: 669739baa7fd582ae7420a3d
paper_authors.name: Jingren Zhou
paper_title: Data-Juicer Sandbox: A Comprehensive Suite for Multimodal Data-Model
    Co-development
paper_summary: The emergence of large-scale multi-modal generative models has drastically
  advanced artificial intelligence, introducing unprecedented levels of
  performance and functionality. However, optimizing these models remains
  challenging due to historically isolated paths of model-centric and
  data-centric developments, leading to suboptimal outcomes and inefficient
  resource utilization. In response, we present a novel sandbox suite tailored
  for integrated data-model co-development. This sandbox provides a comprehensive
  experimental platform, enabling rapid iteration and insight-driven refinement
  of both data and models. Our proposed "Probe-Analyze-Refine" workflow,
  validated through applications on state-of-the-art LLaVA-like and DiT based
  models, yields significant performance boosts, such as topping the VBench
  leaderboard. We also uncover fruitful insights gleaned from exhaustive
  benchmarks, shedding light on the critical interplay between data quality,
  diversity, and model behavior. With the hope of fostering deeper understanding
  and future progress in multi-modal data and generative modeling, our codes,
  datasets, and models are maintained and accessible at
  https://github.com/modelscope/data-juicer/blob/main/docs/Sandbox.md.
paper_ai_summary: A novel sandbox suite for integrated data-model co-development enhances performance in multi-modal generative models, improving data quality and model behavior through a "Probe-Analyze-Refine" workflow.
paper_ai_keywords: multi-modal generative models sandbox suite data-model co-development Probe-Analyze-Refine LLaVA DiT VBench leaderboard