title: DreamClear: High-Capacity Real-World Image Restoration with Privacy-Safe
    Dataset Curation
summary: Image restoration (IR) in real-world scenarios presents significant
  challenges due to the lack of high-capacity models and comprehensive datasets.
  To tackle these issues, we present a dual strategy: GenIR, an innovative data
  curation pipeline, and DreamClear, a cutting-edge Diffusion Transformer
  (DiT)-based image restoration model. GenIR, our pioneering contribution, is a
  dual-prompt learning pipeline that overcomes the limitations of existing
  datasets, which typically comprise only a few thousand images and thus offer
  limited generalizability for larger models. GenIR streamlines the process into
  three stages: image-text pair construction, dual-prompt based fine-tuning, and
  data generation & filtering. This approach circumvents the laborious data
  crawling process, ensuring copyright compliance and providing a cost-effective,
  privacy-safe solution for IR dataset construction. The result is a large-scale
  dataset of one million high-quality images. Our second contribution,
  DreamClear, is a DiT-based image restoration model. It utilizes the generative
  priors of text-to-image (T2I) diffusion models and the robust perceptual
  capabilities of multi-modal large language models (MLLMs) to achieve
  photorealistic restoration. To boost the model's adaptability to diverse
  real-world degradations, we introduce the Mixture of Adaptive Modulator (MoAM).
  It employs token-wise degradation priors to dynamically integrate various
  restoration experts, thereby expanding the range of degradations the model can
  address. Our exhaustive experiments confirm DreamClear's superior performance,
  underlining the efficacy of our dual strategy for real-world image restoration.
  Code and pre-trained models will be available at:
  https://github.com/shallowdream204/DreamClear.
paper_authors._id: 671c719d15313d2c0d58079c
paper_authors.name: Yuang Ai
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-30T09:54:41.552Z
paper_authors._id: 65709464b3501cbcb8f4007e
paper_authors.avatarUrl: /avatars/4fb7524cb21e38138b4c67ff5501b403.svg
paper_authors.fullname: Yuang Ai
paper_authors.type: user
paper_authors.user: shallowdream204
paper_authors._id: 671c719d15313d2c0d58079d
paper_authors.name: Xiaoqiang Zhou
paper_authors._id: 671c719d15313d2c0d58079e
paper_authors.name: Huaibo Huang
paper_authors._id: 671c719d15313d2c0d58079f
paper_authors.name: Xiaotian Han
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-30T09:54:43.229Z
paper_authors._id: 650dde4ce14eeb01d42b37a1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/650dde4ce14eeb01d42b37a1/n5Yv24uofZ2XJjXdYCrKd.png
paper_authors.fullname: Xiaotian Han
paper_authors.type: user
paper_authors.user: xiaotianhan
paper_authors._id: 671c719d15313d2c0d5807a0
paper_authors.name: Zhengyu Chen
paper_authors._id: 671c719d15313d2c0d5807a1
paper_authors.name: Quanzeng You
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-28T10:36:57.630Z
paper_authors._id: 642bb8c87f152f6e72b8d3cd
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/642bb8c87f152f6e72b8d3cd/MXxTssocydbabGZ6XFIic.png
paper_authors.fullname: Quanzeng You
paper_authors.type: user
paper_authors.user: Ye27
paper_authors._id: 671c719d15313d2c0d5807a2
paper_authors.name: Hongxia Yang
paper_title: DreamClear: High-Capacity Real-World Image Restoration with Privacy-Safe
    Dataset Curation
paper_summary: Image restoration (IR) in real-world scenarios presents significant
  challenges due to the lack of high-capacity models and comprehensive datasets.
  To tackle these issues, we present a dual strategy: GenIR, an innovative data
  curation pipeline, and DreamClear, a cutting-edge Diffusion Transformer
  (DiT)-based image restoration model. GenIR, our pioneering contribution, is a
  dual-prompt learning pipeline that overcomes the limitations of existing
  datasets, which typically comprise only a few thousand images and thus offer
  limited generalizability for larger models. GenIR streamlines the process into
  three stages: image-text pair construction, dual-prompt based fine-tuning, and
  data generation & filtering. This approach circumvents the laborious data
  crawling process, ensuring copyright compliance and providing a cost-effective,
  privacy-safe solution for IR dataset construction. The result is a large-scale
  dataset of one million high-quality images. Our second contribution,
  DreamClear, is a DiT-based image restoration model. It utilizes the generative
  priors of text-to-image (T2I) diffusion models and the robust perceptual
  capabilities of multi-modal large language models (MLLMs) to achieve
  photorealistic restoration. To boost the model's adaptability to diverse
  real-world degradations, we introduce the Mixture of Adaptive Modulator (MoAM).
  It employs token-wise degradation priors to dynamically integrate various
  restoration experts, thereby expanding the range of degradations the model can
  address. Our exhaustive experiments confirm DreamClear's superior performance,
  underlining the efficacy of our dual strategy for real-world image restoration.
  Code and pre-trained models will be available at:
  https://github.com/shallowdream204/DreamClear.
paper_ai_summary: A dual strategy involving a data curation pipeline (GenIR) and a DiT-based model (DreamClear) enhances image restoration by creating a large dataset and utilizing generative priors and adaptive modulators.
paper_ai_keywords: Diffusion Transformer (DiT) dual-prompt learning image-text pair construction dual-prompt based fine-tuning data generation & filtering text-to-image (T2I) diffusion models multi-modal large language models (MLLMs) Mixture of Adaptive Modulator (MoAM) token-wise degradation priors