Two Heads are Better Than One: Neural Networks Quantization with 2D Hilbert Curve-based Output Representation
<|reference_start|>Two Heads are Better Than One: Neural Networks Quantization with 2D Hilbert Curve-based Output Representation: Quantization is widely used to increase deep neural networks' (DNN) memory, computation, and power efficiency. Various techniques, such as post-training quantization and quantization-aware training, have been proposed to improve quantization quality. We introduce a novel approach for DNN quantization that uses a redundant representation of DNN's output. We represent the target quantity as a point on a 2D parametric curve. The DNN model is modified to predict 2D points that are mapped back to the target quantity at a post-processing stage. We demonstrate that this mapping can reduce quantization error. For the low-order parametric Hilbert curve, Depth-From-Stereo task, and two models represented by U-Net architecture and vision transformer, we achieved a quantization error reduction by about 5 times for the INT8 model at both CPU and DSP delegates. This gain comes with a minimal inference time increase (less than 7%). Our approach can be applied to other tasks, including segmentation, object detection, and key-points prediction.<|reference_end|>
@article{uss2024two,
    title={Two Heads are Better Than One: Neural Networks Quantization with 2D
  Hilbert Curve-based Output Representation},
    author={Mykhailo Uss, Ruslan Yermolenko, Olena Kolodiazhna, Oleksii Shashko,
  Ivan Safonov, Volodymyr Savin, Yoonjae Yeo, Seowon Ji, and Jaeyun Jeong},
    journal={arXiv preprint arXiv:2405.14024},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.14024},
    primaryClass={cs.CV cs.AI}
}