This paper aims at rapid deployment of the state-of-the-art deep neural networks (DNNs) to energy efficient accelerators without time-consuming fine tuning or the availability of the full datasets. Converting DNNs in full precision to limited precision is essential in taking advantage of the accelerators with reduced memory footprint and computation power. However, such a task is not trivial since it often requires the full training and validation datasets for profiling the network statistics and fine tuning the networks to recover the accuracy lost after quantization. To address these issues, we propose a simple method recognizing channel-level distribution to reduce the quantization-induced accuracy loss and minimize the required image samples for profiling. We evaluated our method on eleven networks trained on the ImageNet classification benchmark and a network trained on the Pascal VOC object detection benchmark. The results prove that the networks can be quantized into 8-bit integer precision without fine tuning.
This paper proposes a technique for channel-wise quantization of CNNs
to 8-bit, fixed point precision. The authors propose several
techniques for analyzing the statistical properties of output channel
activations in order to select the best fractional bit length for each
channel. Experimental results on eleven different CNN architectures
demonstrate that the approaches proposed result in significantly less
accuracy loss when compared to a layer-wise baseline.
The paper has the following strengths:
 1. The experimental results on eleven different architectures (of
    varying depth and breadth) are convincing, and are consistently
    better than layer-wise MAX for choosing fractional bit length.
The paper has the following weak points:
 1. There is not much coherence between the description of the
    approach in section 2.1, Figure 1, Algorithm 1, and
    Figure 2. Notation is used in Algorithm 1 which is never defined.
 2. Related to the previous point, the proposed technique has a lot of
    moving parts and I don't feel that it would be easy to reproduce
    the results of the paper. There are some vague statements, like
    "We resolve this complication by pre-coordinating the fractional
    lengths of the weights", which require significantly more
    precision. This issue -- one of the main issues with channel-wise
    versus layer-wise quantization -- is never returned to in the
    definition of the method.
 3. The experimental comparison with layer-wise quantization is
    somewhat lacking. Is layer-wise MAX the state-of-the-art in CNN
    quantization? The results comparing channel-wise and layer-wise
    MAX are already convincing, but are the moment-analysis approaches
    not equally applicable to layer-wise quantization?
    State-of-the-art results that are less sensitive to outliers
    should be included in Table 1. A comparison with layer-wise
    approaches would be nice to have also in Figure 4 to show
    sensitivity to profiling set size.
The experimental results in the paper are impressive, and the analysis
motivating the approach is convincing. However, there are presentation
and clarity issues in the technical development, and the comparative
analysis is lacking broader comparisons with the state-of-the-art (to
be fair, the authors recognize that layer-wise MAX as a baseline is
particularly susceptible to outliers). These two aspects combined,
however, lead me to the opinion that this work is just not quite ready
for publication at ICLR.