ZipIR: Latent Pyramid Diffusion Transformer for High-Resolution Image
  Restoration
Recent progress in generative models has significantly improved image
restoration capabilities, particularly through powerful diffusion models that
offer remarkable recovery of semantic details and local fidelity. However,
deploying these models at ultra-high resolutions faces a critical trade-off
between quality and efficiency due to the computational demands of long-range
attention mechanisms. To address this, we introduce ZipIR, a novel framework
that enhances efficiency, scalability, and long-range modeling for high-res
image restoration. ZipIR employs a highly compressed latent representation that
compresses image 32x, effectively reducing the number of spatial tokens, and
enabling the use of high-capacity models like the Diffusion Transformer (DiT).
Toward this goal, we propose a Latent Pyramid VAE (LP-VAE) design that
structures the latent space into sub-bands to ease diffusion training. Trained
on full images up to 2K resolution, ZipIR surpasses existing diffusion-based
methods, offering unmatched speed and quality in restoring high-resolution
images from severely degraded inputs.
67fd2584b12728f2678f7dfb
Yongsheng Yu
claimed_verified
2025-04-14T18:39:46.656Z
635b6bdfd63bcb528acefa78
/avatars/54c62b41c147d0450bdfc72121860bb4.svg
Yongsheng Yu
user
yeates
67fd2584b12728f2678f7dfc
Haitian Zheng
67fd2584b12728f2678f7dfd
Zhifei Zhang
67fd2584b12728f2678f7dfe
Jianming Zhang
67fd2584b12728f2678f7dff
Yuqian Zhou
67fd2584b12728f2678f7e00
Connelly Barnes
67fd2584b12728f2678f7e01
Yuchen Liu
67fd2584b12728f2678f7e02
Wei Xiong
67fd2584b12728f2678f7e03
Zhe Lin
67fd2584b12728f2678f7e04
Jiebo Luo
ZipIR: Latent Pyramid Diffusion Transformer for High-Resolution Image
  Restoration
Recent progress in generative models has significantly improved image
restoration capabilities, particularly through powerful diffusion models that
offer remarkable recovery of semantic details and local fidelity. However,
deploying these models at ultra-high resolutions faces a critical trade-off
between quality and efficiency due to the computational demands of long-range
attention mechanisms. To address this, we introduce ZipIR, a novel framework
that enhances efficiency, scalability, and long-range modeling for high-res
image restoration. ZipIR employs a highly compressed latent representation that
compresses image 32x, effectively reducing the number of spatial tokens, and
enabling the use of high-capacity models like the Diffusion Transformer (DiT).
Toward this goal, we propose a Latent Pyramid VAE (LP-VAE) design that
structures the latent space into sub-bands to ease diffusion training. Trained
on full images up to 2K resolution, ZipIR surpasses existing diffusion-based
methods, offering unmatched speed and quality in restoring high-resolution
images from severely degraded inputs.
ZipIR, a novel framework, uses a compressed latent space and Latent Pyramid VAE to enhance efficiency and scalability of diffusion models for high-resolution image restoration.
diffusion models semantic details long-range attention mechanisms ZipIR latent representation spatial tokens Diffusion Transformer Latent Pyramid VAE high-resolution image restoration severely degraded inputs