Quantization reduces computation costs of neural networks but suffers from performance degeneration. Is this accuracy drop due to the reduced capacity, or inefficient training during the quantization procedure? After looking into the gradient propagation process of neural networks by viewing the weights and intermediate activations as random variables, we discover two critical rules for efficient training. Recent quantization approaches violates the two rules and results in degenerated convergence. To deal with this problem, we propose a simple yet effective technique, named scale-adjusted training (SAT), to comply with the discovered rules and facilitates efficient training. We also analyze the quantization error introduced in calculating the gradient in the popular parameterized clipping activation (PACT) technique. Through SAT together with gradient-calibrated PACT, quantized models obtain comparable or even better performance than their full-precision counterparts, achieving state-of-the-art accuracy with consistent improvement over previous quantization methods on a wide spectrum of models including MobileNet-V1/V2 and PreResNet-50.
Quick Summary: Based on semi-quantitative analysis the paper first proposes two rules for quantization of DNNs, then extends previous methods based on these rules to propose specific technique for quantizing activations and weights. Experimental results are provided on MobileNet V1/VB2 and ResNet50 and compare favourably against baselines (which are old and unfortunately do not represent recent developments in the literature). Overall I found novelty low and also there are several recent papers already published before this submission which provide results which are at least as good or even better
Details
Specifically the rules are: 
1. To prevent logits from entering saturation region of the cross entropy loss, the effective weight in the last fully connected layer should be small.  
2.To keep the gradient of weights in the same scale across the whole network, either BN layers should be used after linear layers such as convolution and fully- connected layers, or the variance of the effective weights should be on the order of the reciprocal of the number of neurons of the linear layer (n_l).
They use DoReFa for weight quantization and PACT for activation quantization based on the above observations/rules. However the contributions were not really very novel in my opinion. See especially the paper "LEARNED STEP SIZE QUANTIZATION" I linked below when explaining novelty
This reviewer feels that the authors were perhaps unaware of several important contributions to the literature this year which are substantially better than the baselines compared against in the paper. I list a few below for the authors to compare against, and to explain their novelty in contributions in the rebuttal phase. Unfortunately this makes me feel the paper should be a clear reject in its current state, unless the authors can convince us otherwise. 
https://arxiv.org/pdf/1902.08153.pdf
https://arxiv.org/pdf/1903.08066.pdf
https://arxiv.org/pdf/1905.11452.pdf
https://arxiv.org/pdf/1905.13082.pdf