FLIQS: One-Shot Mixed-Precision Floating-Point and Integer Quantization
  Search
Quantization has become a mainstream compression technique for reducing model
size, computational requirements, and energy consumption for modern deep neural
networks (DNNs). With the improved numerical support in recent hardware,
including multiple variants of integer and floating point, mixed-precision
quantization has become necessary to achieve high-quality results with low
model cost. Prior mixed-precision quantization methods have performed a
post-training quantization search, which compromises on accuracy, or a
differentiable quantization search, which leads to high memory usage from
branching. Therefore, we propose the first one-shot mixed-precision
quantization search that eliminates the need for retraining in both integer and
low-precision floating point models. We evaluate our floating-point and integer
quantization search (FLIQS) on multiple convolutional networks and vision
transformer models to discover Pareto-optimal models. Our approach discovers
models that improve upon uniform precision, manual mixed-precision, and recent
integer quantization search methods. With the proposed integer quantization
search, we increase the accuracy of ResNet-18 on ImageNet by 1.31% points and
ResNet-50 by 0.90% points with equivalent model cost over previous methods.
Additionally, for the first time, we explore a novel mixed-precision
floating-point search and improve MobileNetV2 by up to 0.98% points compared to
prior state-of-the-art FP8 models. Finally, we extend FLIQS to simultaneously
search a joint quantization and neural architecture space and improve the
ImageNet accuracy by 2.69% points with similar model cost on a MobileNetV2
search space.
64d1d8ca995a0b5d59aa8487
Jordan Dotzel
admin_assigned
2023-08-08T15:07:42.775Z
640578e81a3babee78e814c0
https://cdn-avatars.huggingface.co/v1/production/uploads/640578e81a3babee78e814c0/hCqM3XCkbFCK1lVjsxWoe.jpeg
Jordan Dotzel
user
jordandotzel
64d1d8ca995a0b5d59aa8488
Gang Wu
64d1d8ca995a0b5d59aa8489
Andrew Li
64d1d8ca995a0b5d59aa848a
Muhammad Umar
64d1d8ca995a0b5d59aa848b
Yun Ni
64d1d8ca995a0b5d59aa848c
Mohamed S. Abdelfattah
admin_assigned
2023-08-08T15:09:23.187Z
60cf9f0143c9305fa34dd882
https://cdn-avatars.huggingface.co/v1/production/uploads/1624219368078-noauth.jpeg
Mohamed Abdelfattah
user
mohsaied
64d1d8ca995a0b5d59aa848d
Zhiru Zhang
64d1d8ca995a0b5d59aa848e
Liqun Cheng
64d1d8ca995a0b5d59aa848f
Martin G. Dixon
64d1d8ca995a0b5d59aa8490
Norman P. Jouppi
64d1d8ca995a0b5d59aa8491
Quoc V. Le
64d1d8ca995a0b5d59aa8492
Sheng Li
FLIQS: One-Shot Mixed-Precision Floating-Point and Integer Quantization
  Search
Quantization has become a mainstream compression technique for reducing model
size, computational requirements, and energy consumption for modern deep neural
networks (DNNs). With the improved numerical support in recent hardware,
including multiple variants of integer and floating point, mixed-precision
quantization has become necessary to achieve high-quality results with low
model cost. Prior mixed-precision quantization methods have performed a
post-training quantization search, which compromises on accuracy, or a
differentiable quantization search, which leads to high memory usage from
branching. Therefore, we propose the first one-shot mixed-precision
quantization search that eliminates the need for retraining in both integer and
low-precision floating point models. We evaluate our floating-point and integer
quantization search (FLIQS) on multiple convolutional networks and vision
transformer models to discover Pareto-optimal models. Our approach discovers
models that improve upon uniform precision, manual mixed-precision, and recent
integer quantization search methods. With the proposed integer quantization
search, we increase the accuracy of ResNet-18 on ImageNet by 1.31% points and
ResNet-50 by 0.90% points with equivalent model cost over previous methods.
Additionally, for the first time, we explore a novel mixed-precision
floating-point search and improve MobileNetV2 by up to 0.98% points compared to
prior state-of-the-art FP8 models. Finally, we extend FLIQS to simultaneously
search a joint quantization and neural architecture space and improve the
ImageNet accuracy by 2.69% points with similar model cost on a MobileNetV2
search space.
FLIQS, a one-shot mixed-precision quantization search, improves accuracy and efficiency in deep neural network models without retraining.
quantization mixed-precision quantization post-training quantization differentiable quantization floating-point integer one-shot quantization ResNet-18 ResNet-50 MobileNetV2 vision transformers Pareto-optimal models neon architecture search