title: Diffusion Curriculum: Synthetic-to-Real Generative Curriculum Learning
    via Image-Guided Diffusion
summary: Low-quality or scarce data has posed significant challenges for training deep
  neural networks in practice. While classical data augmentation cannot
  contribute very different new data, diffusion models opens up a new door to
  build self-evolving AI by generating high-quality and diverse synthetic data
  through text-guided prompts. However, text-only guidance cannot control
  synthetic images' proximity to the original images, resulting in
  out-of-distribution data detrimental to the model performance. To overcome the
  limitation, we study image guidance to achieve a spectrum of interpolations
  between synthetic and real images. With stronger image guidance, the generated
  images are similar to the training data but hard to learn. While with weaker
  image guidance, the synthetic images will be easier for model but contribute to
  a larger distribution gap with the original data. The generated full spectrum
  of data enables us to build a novel "Diffusion Curriculum (DisCL)". DisCL
  adjusts the image guidance level of image synthesis for each training stage: It
  identifies and focuses on hard samples for the model and assesses the most
  effective guidance level of synthetic images to improve hard data learning. We
  apply DisCL to two challenging tasks: long-tail (LT) classification and
  learning from low-quality data. It focuses on lower-guidance images of
  high-quality to learn prototypical features as a warm-up of learning
  higher-guidance images that might be weak on diversity or quality. Extensive
  experiments showcase a gain of 2.7% and 2.1% in OOD and ID macro-accuracy when
  applying DisCL to iWildCam dataset. On ImageNet-LT, DisCL improves the base
  model's tail-class accuracy from 4.4% to 23.64% and leads to a 4.02%
  improvement in all-class accuracy.
paper_authors._id: 6711d1dbff45f0171cde152c
paper_authors.name: Yijun Liang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-22T08:01:54.933Z
paper_authors._id: 66720ab819bebc69b5b93685
paper_authors.avatarUrl: /avatars/b2f1314d9a26f6f5eaf6cebdb0d28812.svg
paper_authors.fullname: Joanna Liang
paper_authors.type: user
paper_authors.user: joliang17
paper_authors._id: 6711d1dbff45f0171cde152d
paper_authors.name: Shweta Bhardwaj
paper_authors._id: 6711d1dbff45f0171cde152e
paper_authors.name: Tianyi Zhou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-18T08:12:16.307Z
paper_authors._id: 647f5af5b0e96764589f3b2a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/VJ4cDyjp5M3V5WmI5gPIU.jpeg
paper_authors.fullname: Tianyi Zhou
paper_authors.type: user
paper_authors.user: zhoutianyi
paper_title: Diffusion Curriculum: Synthetic-to-Real Generative Curriculum Learning
    via Image-Guided Diffusion
paper_summary: Low-quality or scarce data has posed significant challenges for training deep
  neural networks in practice. While classical data augmentation cannot
  contribute very different new data, diffusion models opens up a new door to
  build self-evolving AI by generating high-quality and diverse synthetic data
  through text-guided prompts. However, text-only guidance cannot control
  synthetic images' proximity to the original images, resulting in
  out-of-distribution data detrimental to the model performance. To overcome the
  limitation, we study image guidance to achieve a spectrum of interpolations
  between synthetic and real images. With stronger image guidance, the generated
  images are similar to the training data but hard to learn. While with weaker
  image guidance, the synthetic images will be easier for model but contribute to
  a larger distribution gap with the original data. The generated full spectrum
  of data enables us to build a novel "Diffusion Curriculum (DisCL)". DisCL
  adjusts the image guidance level of image synthesis for each training stage: It
  identifies and focuses on hard samples for the model and assesses the most
  effective guidance level of synthetic images to improve hard data learning. We
  apply DisCL to two challenging tasks: long-tail (LT) classification and
  learning from low-quality data. It focuses on lower-guidance images of
  high-quality to learn prototypical features as a warm-up of learning
  higher-guidance images that might be weak on diversity or quality. Extensive
  experiments showcase a gain of 2.7% and 2.1% in OOD and ID macro-accuracy when
  applying DisCL to iWildCam dataset. On ImageNet-LT, DisCL improves the base
  model's tail-class accuracy from 4.4% to 23.64% and leads to a 4.02%
  improvement in all-class accuracy.
paper_ai_summary: A novel diffusion curriculum dynamically adjusts image guidance to improve learning from low-quality or scarce data, enhancing model accuracy on both in-distribution and out-of-distribution tasks.
paper_ai_keywords: diffusion models text-guided prompts image guidance synthetic data Diffusion Curriculum (DisCL) long-tail classification learning from low-quality data prototypical features OOD accuracy ID accuracy