Efficient Quantization Strategies for Latent Diffusion Models
Latent Diffusion Models (LDMs) capture the dynamic evolution of latent
variables over time, blending patterns and multimodality in a generative
system. Despite the proficiency of LDM in various applications, such as
text-to-image generation, facilitated by robust text encoders and a variational
autoencoder, the critical need to deploy large generative models on edge
devices compels a search for more compact yet effective alternatives. Post
Training Quantization (PTQ), a method to compress the operational size of deep
learning models, encounters challenges when applied to LDM due to temporal and
structural complexities. This study proposes a quantization strategy that
efficiently quantize LDMs, leveraging Signal-to-Quantization-Noise Ratio (SQNR)
as a pivotal metric for evaluation. By treating the quantization discrepancy as
relative noise and identifying sensitive part(s) of a model, we propose an
efficient quantization approach encompassing both global and local strategies.
The global quantization process mitigates relative quantization noise by
initiating higher-precision quantization on sensitive blocks, while local
treatments address specific challenges in quantization-sensitive and
time-sensitive modules. The outcomes of our experiments reveal that the
implementation of both global and local treatments yields a highly efficient
and effective Post Training Quantization (PTQ) of LDMs.
6577f2903ceeb2f078f909a0
Yuewei Yang
admin_assigned
2023-12-12T08:36:27.713Z
64e61822204192b8d65f1c57
/avatars/7cc04dfc63c1c3f2012c5cb1334d064a.svg
Yuewei Yang
user
yy353
6577f2903ceeb2f078f909a1
Xiaoliang Dai
admin_assigned
2023-12-12T08:36:33.704Z
6549417b3ce45eb764faf993
/avatars/d310f475d0697f5f13b3d4141ea0ccaf.svg
Xiaoliang Dai
user
daixl1992
6577f2903ceeb2f078f909a2
Jialiang Wang
admin_assigned
2023-12-12T08:36:39.400Z
65300c8a5da070af779cf047
/avatars/8e24a2a1d7a8ced05a4c8e25b3d1be5a.svg
Jialiang Wang
user
jwhuggingf
6577f2903ceeb2f078f909a3
Peizhao Zhang
admin_assigned
2023-12-12T08:36:46.079Z
6433a635a5aed21dd110381e
/avatars/b1fa4fe1c8bd7989875233820fa425af.svg
Peizhao Zhang
user
stzpz
6577f2903ceeb2f078f909a4
Hongbo Zhang
admin_assigned
2023-12-12T08:37:10.085Z
6182ac4df82814e44d033d51
/avatars/ff56e4a0654388be9b4c052237d8e3cb.svg
Hongbo Zhang
user
StevenZHB
Efficient Quantization Strategies for Latent Diffusion Models
Latent Diffusion Models (LDMs) capture the dynamic evolution of latent
variables over time, blending patterns and multimodality in a generative
system. Despite the proficiency of LDM in various applications, such as
text-to-image generation, facilitated by robust text encoders and a variational
autoencoder, the critical need to deploy large generative models on edge
devices compels a search for more compact yet effective alternatives. Post
Training Quantization (PTQ), a method to compress the operational size of deep
learning models, encounters challenges when applied to LDM due to temporal and
structural complexities. This study proposes a quantization strategy that
efficiently quantize LDMs, leveraging Signal-to-Quantization-Noise Ratio (SQNR)
as a pivotal metric for evaluation. By treating the quantization discrepancy as
relative noise and identifying sensitive part(s) of a model, we propose an
efficient quantization approach encompassing both global and local strategies.
The global quantization process mitigates relative quantization noise by
initiating higher-precision quantization on sensitive blocks, while local
treatments address specific challenges in quantization-sensitive and
time-sensitive modules. The outcomes of our experiments reveal that the
implementation of both global and local treatments yields a highly efficient
and effective Post Training Quantization (PTQ) of LDMs.
A quantization strategy leveraging global and local treatments efficiently compresses Latent Diffusion Models (LDMs) for edge deployment, maintaining model performance.
Latent Diffusion Models LDMs Post Training Quantization PTQ Signal-to-Quantization-Noise Ratio SQNR quantization discrepancy quantization-sensitive time-sensitive modules