Training-Free Large Model Priors for Multiple-in-One Image Restoration
<|reference_start|>Training-Free Large Model Priors for Multiple-in-One Image Restoration: Image restoration aims to reconstruct the latent clear images from their degraded versions. Despite the notable achievement, existing methods predominantly focus on handling specific degradation types and thus require specialized models, impeding real-world applications in dynamic degradation scenarios. To address this issue, we propose Large Model Driven Image Restoration framework (LMDIR), a novel multiple-in-one image restoration paradigm that leverages the generic priors from large multi-modal language models (MMLMs) and the pretrained diffusion models. In detail, LMDIR integrates three key prior knowledges: 1) global degradation knowledge from MMLMs, 2) scene-aware contextual descriptions generated by MMLMs, and 3) fine-grained high-quality reference images synthesized by diffusion models guided by MMLM descriptions. Standing on above priors, our architecture comprises a query-based prompt encoder, degradation-aware transformer block injecting global degradation knowledge, content-aware transformer block incorporating scene description, and reference-based transformer block incorporating fine-grained image priors. This design facilitates single-stage training paradigm to address various degradations while supporting both automatic and user-guided restoration. Extensive experiments demonstrate that our designed method outperforms state-of-the-art competitors on multiple evaluation benchmarks.<|reference_end|>
@article{he2024training-free,
    title={Training-Free Large Model Priors for Multiple-in-One Image Restoration},
    author={Xuanhua He, Lang Li, Yingying Wang, Hui Zheng, Ke Cao, Keyu Yan, Rui
  Li, Chengjun Xie, Jie Zhang, Man Zhou},
    journal={arXiv preprint arXiv:2407.13181},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.13181},
    primaryClass={cs.CV}
}