title: LiteVAE: Lightweight and Efficient Variational Autoencoders for Latent
    Diffusion Models
summary: Advances in latent diffusion models (LDMs) have revolutionized
  high-resolution image generation, but the design space of the autoencoder that
  is central to these systems remains underexplored. In this paper, we introduce
  LiteVAE, a family of autoencoders for LDMs that leverage the 2D discrete
  wavelet transform to enhance scalability and computational efficiency over
  standard variational autoencoders (VAEs) with no sacrifice in output quality.
  We also investigate the training methodologies and the decoder architecture of
  LiteVAE and propose several enhancements that improve the training dynamics and
  reconstruction quality. Our base LiteVAE model matches the quality of the
  established VAEs in current LDMs with a six-fold reduction in encoder
  parameters, leading to faster training and lower GPU memory requirements, while
  our larger model outperforms VAEs of comparable complexity across all evaluated
  metrics (rFID, LPIPS, PSNR, and SSIM).
paper_authors._id: 665011a5a35b05faab595617
paper_authors.name: Seyedmorteza Sadat
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-05-24T09:31:39.850Z
paper_authors._id: 63b4b02a103617b0a5b0ee2e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63b4b02a103617b0a5b0ee2e/LasBC-pCnPJ6XuCt6qqcp.jpeg
paper_authors.fullname: Seyedmorteza Sadat
paper_authors.type: user
paper_authors.user: msadat97
paper_authors._id: 665011a5a35b05faab595618
paper_authors.name: Jakob Buhmann
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-24T11:04:09.083Z
paper_authors._id: 62013227611d68ef490683a2
paper_authors.avatarUrl: /avatars/098e8a644692b2a9df052ceea462cbf6.svg
paper_authors.fullname: Jakob Buhmann
paper_authors.type: user
paper_authors.user: jjbuhmann
paper_authors._id: 665011a5a35b05faab595619
paper_authors.name: Derek Bradley
paper_authors._id: 665011a5a35b05faab59561a
paper_authors.name: Otmar Hilliges
paper_authors._id: 665011a5a35b05faab59561b
paper_authors.name: Romann M. Weber
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-24T11:04:30.751Z
paper_authors._id: 630f7646197cd3f24e7f8e9f
paper_authors.avatarUrl: /avatars/59bbd4ed38277b313051aac78f6808ac.svg
paper_authors.fullname: Romann Weber
paper_authors.type: user
paper_authors.user: RMW
paper_title: LiteVAE: Lightweight and Efficient Variational Autoencoders for Latent
    Diffusion Models
paper_summary: Advances in latent diffusion models (LDMs) have revolutionized
  high-resolution image generation, but the design space of the autoencoder that
  is central to these systems remains underexplored. In this paper, we introduce
  LiteVAE, a family of autoencoders for LDMs that leverage the 2D discrete
  wavelet transform to enhance scalability and computational efficiency over
  standard variational autoencoders (VAEs) with no sacrifice in output quality.
  We also investigate the training methodologies and the decoder architecture of
  LiteVAE and propose several enhancements that improve the training dynamics and
  reconstruction quality. Our base LiteVAE model matches the quality of the
  established VAEs in current LDMs with a six-fold reduction in encoder
  parameters, leading to faster training and lower GPU memory requirements, while
  our larger model outperforms VAEs of comparable complexity across all evaluated
  metrics (rFID, LPIPS, PSNR, and SSIM).
paper_ai_summary: LiteVAE, an autoencoder family for LDMs using the 2D discrete wavelet transform, enhances scalability and efficiency without compromising quality, offering improved training dynamics and reconstruction quality.
paper_ai_keywords: latent diffusion models LiteVAE 2D discrete wavelet transform variational autoencoders training dynamics rFID LPIPS PSNR SSIM