title: FLIQS: One-Shot Mixed-Precision Floating-Point and Integer Quantization
    Search
summary: Quantization has become a mainstream compression technique for reducing model
  size, computational requirements, and energy consumption for modern deep neural
  networks (DNNs). With the improved numerical support in recent hardware,
  including multiple variants of integer and floating point, mixed-precision
  quantization has become necessary to achieve high-quality results with low
  model cost. Prior mixed-precision quantization methods have performed a
  post-training quantization search, which compromises on accuracy, or a
  differentiable quantization search, which leads to high memory usage from
  branching. Therefore, we propose the first one-shot mixed-precision
  quantization search that eliminates the need for retraining in both integer and
  low-precision floating point models. We evaluate our floating-point and integer
  quantization search (FLIQS) on multiple convolutional networks and vision
  transformer models to discover Pareto-optimal models. Our approach discovers
  models that improve upon uniform precision, manual mixed-precision, and recent
  integer quantization search methods. With the proposed integer quantization
  search, we increase the accuracy of ResNet-18 on ImageNet by 1.31% points and
  ResNet-50 by 0.90% points with equivalent model cost over previous methods.
  Additionally, for the first time, we explore a novel mixed-precision
  floating-point search and improve MobileNetV2 by up to 0.98% points compared to
  prior state-of-the-art FP8 models. Finally, we extend FLIQS to simultaneously
  search a joint quantization and neural architecture space and improve the
  ImageNet accuracy by 2.69% points with similar model cost on a MobileNetV2
  search space.
paper_authors._id: 64d1d8ca995a0b5d59aa8487
paper_authors.name: Jordan Dotzel
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-08T15:07:42.775Z
paper_authors._id: 640578e81a3babee78e814c0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/640578e81a3babee78e814c0/hCqM3XCkbFCK1lVjsxWoe.jpeg
paper_authors.fullname: Jordan Dotzel
paper_authors.type: user
paper_authors.user: jordandotzel
paper_authors._id: 64d1d8ca995a0b5d59aa8488
paper_authors.name: Gang Wu
paper_authors._id: 64d1d8ca995a0b5d59aa8489
paper_authors.name: Andrew Li
paper_authors._id: 64d1d8ca995a0b5d59aa848a
paper_authors.name: Muhammad Umar
paper_authors._id: 64d1d8ca995a0b5d59aa848b
paper_authors.name: Yun Ni
paper_authors._id: 64d1d8ca995a0b5d59aa848c
paper_authors.name: Mohamed S. Abdelfattah
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-08T15:09:23.187Z
paper_authors._id: 60cf9f0143c9305fa34dd882
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1624219368078-noauth.jpeg
paper_authors.fullname: Mohamed Abdelfattah
paper_authors.type: user
paper_authors.user: mohsaied
paper_authors._id: 64d1d8ca995a0b5d59aa848d
paper_authors.name: Zhiru Zhang
paper_authors._id: 64d1d8ca995a0b5d59aa848e
paper_authors.name: Liqun Cheng
paper_authors._id: 64d1d8ca995a0b5d59aa848f
paper_authors.name: Martin G. Dixon
paper_authors._id: 64d1d8ca995a0b5d59aa8490
paper_authors.name: Norman P. Jouppi
paper_authors._id: 64d1d8ca995a0b5d59aa8491
paper_authors.name: Quoc V. Le
paper_authors._id: 64d1d8ca995a0b5d59aa8492
paper_authors.name: Sheng Li
paper_title: FLIQS: One-Shot Mixed-Precision Floating-Point and Integer Quantization
    Search
paper_summary: Quantization has become a mainstream compression technique for reducing model
  size, computational requirements, and energy consumption for modern deep neural
  networks (DNNs). With the improved numerical support in recent hardware,
  including multiple variants of integer and floating point, mixed-precision
  quantization has become necessary to achieve high-quality results with low
  model cost. Prior mixed-precision quantization methods have performed a
  post-training quantization search, which compromises on accuracy, or a
  differentiable quantization search, which leads to high memory usage from
  branching. Therefore, we propose the first one-shot mixed-precision
  quantization search that eliminates the need for retraining in both integer and
  low-precision floating point models. We evaluate our floating-point and integer
  quantization search (FLIQS) on multiple convolutional networks and vision
  transformer models to discover Pareto-optimal models. Our approach discovers
  models that improve upon uniform precision, manual mixed-precision, and recent
  integer quantization search methods. With the proposed integer quantization
  search, we increase the accuracy of ResNet-18 on ImageNet by 1.31% points and
  ResNet-50 by 0.90% points with equivalent model cost over previous methods.
  Additionally, for the first time, we explore a novel mixed-precision
  floating-point search and improve MobileNetV2 by up to 0.98% points compared to
  prior state-of-the-art FP8 models. Finally, we extend FLIQS to simultaneously
  search a joint quantization and neural architecture space and improve the
  ImageNet accuracy by 2.69% points with similar model cost on a MobileNetV2
  search space.
paper_ai_keywords: quantization mixed-precision quantization post-training quantization search differentiable quantization search one-shot mixed-precision quantization search floating-point and integer quantization search (FLIQS) convolutional networks vision transformer models Pareto-optimal models uniform precision manual mixed-precision integer quantization search ResNet-18 ImageNet MobileNetV2 FP8 models joint quantization and neural architecture space