DiTAS: Quantizing Diffusion Transformers via Enhanced Activation Smoothing
<|reference_start|>DiTAS: Quantizing Diffusion Transformers via Enhanced Activation Smoothing: Diffusion Transformers (DiTs) have recently attracted significant interest from both industry and academia due to their enhanced capabilities in visual generation, surpassing the performance of traditional diffusion models that employ U-Net. However, the improved performance of DiTs comes at the expense of higher parameter counts and implementation costs, which significantly limits their deployment on resource-constrained devices like mobile phones. We propose DiTAS, a data-free post-training quantization (PTQ) method for efficient DiT inference. DiTAS relies on the proposed temporal-aggregated smoothing techniques to mitigate the impact of the channel-wise outliers within the input activations, leading to much lower quantization error under extremely low bitwidth. To further enhance the performance of the quantized DiT, we adopt the layer-wise grid search strategy to optimize the smoothing factor. Experimental results demonstrate that our approach enables 4-bit weight, 8-bit activation (W4A8) quantization for DiTs while maintaining comparable performance as the full-precision model.<|reference_end|>
@article{dong2024ditas:,
    title={DiTAS: Quantizing Diffusion Transformers via Enhanced Activation
  Smoothing},
    author={Zhenyuan Dong, Sai Qian Zhang},
    journal={arXiv preprint arXiv:2409.07756},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.07756},
    primaryClass={cs.CV}
}