Adaptive quantization with mixed-precision based on low-cost proxy
<|reference_start|>Adaptive quantization with mixed-precision based on low-cost proxy: It is critical to deploy complicated neural network models on hardware with limited resources. This paper proposes a novel model quantization method, named the Low-Cost Proxy-Based Adaptive Mixed-Precision Model Quantization (LCPAQ), which contains three key modules. The hardware-aware module is designed by considering the hardware limitations, while an adaptive mixed-precision quantization module is developed to evaluate the quantization sensitivity by using the Hessian matrix and Pareto frontier techniques. Integer linear programming is used to fine-tune the quantization across different layers. Then the low-cost proxy neural architecture search module efficiently explores the ideal quantization hyperparameters. Experiments on the ImageNet demonstrate that the proposed LCPAQ achieves comparable or superior quantization accuracy to existing mixed-precision models. Notably, LCPAQ achieves 1/200 of the search time compared with existing methods, which provides a shortcut in practical quantization use for resource-limited devices.<|reference_end|>
@article{chen2024adaptive,
    title={Adaptive quantization with mixed-precision based on low-cost proxy},
    author={Junzhe Chen, Qiao Yang, Senmao Tian, Shunli Zhang},
    journal={ICASSP2024},
    year={2024},
    doi={10.1109/ICASSP48485.2024.10447866},
    archivePrefix={arXiv},
    eprint={2402.17706},
    primaryClass={cs.CV}
}