title: ZipIR: Latent Pyramid Diffusion Transformer for High-Resolution Image
    Restoration
summary: Recent progress in generative models has significantly improved image
  restoration capabilities, particularly through powerful diffusion models that
  offer remarkable recovery of semantic details and local fidelity. However,
  deploying these models at ultra-high resolutions faces a critical trade-off
  between quality and efficiency due to the computational demands of long-range
  attention mechanisms. To address this, we introduce ZipIR, a novel framework
  that enhances efficiency, scalability, and long-range modeling for high-res
  image restoration. ZipIR employs a highly compressed latent representation that
  compresses image 32x, effectively reducing the number of spatial tokens, and
  enabling the use of high-capacity models like the Diffusion Transformer (DiT).
  Toward this goal, we propose a Latent Pyramid VAE (LP-VAE) design that
  structures the latent space into sub-bands to ease diffusion training. Trained
  on full images up to 2K resolution, ZipIR surpasses existing diffusion-based
  methods, offering unmatched speed and quality in restoring high-resolution
  images from severely degraded inputs.
paper_authors._id: 67fd2584b12728f2678f7dfb
paper_authors.name: Yongsheng Yu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-14T18:39:46.656Z
paper_authors._id: 635b6bdfd63bcb528acefa78
paper_authors.avatarUrl: /avatars/54c62b41c147d0450bdfc72121860bb4.svg
paper_authors.fullname: Yongsheng Yu
paper_authors.type: user
paper_authors.user: yeates
paper_authors._id: 67fd2584b12728f2678f7dfc
paper_authors.name: Haitian Zheng
paper_authors._id: 67fd2584b12728f2678f7dfd
paper_authors.name: Zhifei Zhang
paper_authors._id: 67fd2584b12728f2678f7dfe
paper_authors.name: Jianming Zhang
paper_authors._id: 67fd2584b12728f2678f7dff
paper_authors.name: Yuqian Zhou
paper_authors._id: 67fd2584b12728f2678f7e00
paper_authors.name: Connelly Barnes
paper_authors._id: 67fd2584b12728f2678f7e01
paper_authors.name: Yuchen Liu
paper_authors._id: 67fd2584b12728f2678f7e02
paper_authors.name: Wei Xiong
paper_authors._id: 67fd2584b12728f2678f7e03
paper_authors.name: Zhe Lin
paper_authors._id: 67fd2584b12728f2678f7e04
paper_authors.name: Jiebo Luo
paper_title: ZipIR: Latent Pyramid Diffusion Transformer for High-Resolution Image
    Restoration
paper_summary: Recent progress in generative models has significantly improved image
  restoration capabilities, particularly through powerful diffusion models that
  offer remarkable recovery of semantic details and local fidelity. However,
  deploying these models at ultra-high resolutions faces a critical trade-off
  between quality and efficiency due to the computational demands of long-range
  attention mechanisms. To address this, we introduce ZipIR, a novel framework
  that enhances efficiency, scalability, and long-range modeling for high-res
  image restoration. ZipIR employs a highly compressed latent representation that
  compresses image 32x, effectively reducing the number of spatial tokens, and
  enabling the use of high-capacity models like the Diffusion Transformer (DiT).
  Toward this goal, we propose a Latent Pyramid VAE (LP-VAE) design that
  structures the latent space into sub-bands to ease diffusion training. Trained
  on full images up to 2K resolution, ZipIR surpasses existing diffusion-based
  methods, offering unmatched speed and quality in restoring high-resolution
  images from severely degraded inputs.
paper_ai_summary: ZipIR, a novel framework, uses a compressed latent space and Latent Pyramid VAE to enhance efficiency and scalability of diffusion models for high-resolution image restoration.
paper_ai_keywords: diffusion models semantic details long-range attention mechanisms ZipIR latent representation spatial tokens Diffusion Transformer Latent Pyramid VAE high-resolution image restoration severely degraded inputs