Quantizing weights and activations of deep neural networks results in significant improvement in inference efficiency at the cost of lower accuracy. A source of the accuracy gap between full precision and quantized models is the quantization error. In this work, we focus on the binary quantization, in which values are mapped to -1 and 1. We introduce several novel quantization algorithms: optimal 1-bit, ternary, 2-bits, and greedy. Our quantization algorithms can be implemented efficiently on the hardware using bitwise operations. We present proofs to show that our proposed methods are optimal, and also provide empirical error analysis. We conduct experiments on the ImageNet dataset and show a reduced accuracy gap when using the proposed optimal quantization algorithms.
This paper proposes to quantize the weights of neural networks that can minimize the L_2 loss between the quantized values and the full-precision ones. The authors propose solutions for optimal 1-bit/ternary/2-bit quantization, as well as a greedy algorithm to approximate the optimal k-bit quantization.  Experiments are performed on image classification data set ImageNet using ResNet-18.
First of all, the authors should explicitly state in the paper that the optimality is in terms of what?  Indeed the authors obtain the solution of (1) quantization with a scaling parameter, and (2) in terms of minimizing quantization error (L_2 loss between the quantized value and the full-precision one), which is quite restricted to be a universally optimal one.
Since the number of weights and activations are limited in the network, it is not appropriate to formulate quantization error the weights and activations in the format of continuous distribution in (4) and (8). 
It is also not clear to me why this paper begins with rank-1 quantization but ends up with scaled quantization. What kind of assumption are used in this approximation, and can the optimality still be guaranteed?
One of my main concerns is the novelty of this paper. Many of the solutions in the paper have already been discovered in literature. For instance, the optimal 1-bit solution in (5) was already obtained in Binary-Weight-Network [1] in 2016. The optimal ternary solution (i.e., the ternarization threshold should be 1/2 of the scaling parameter) in (12)  was also already obtained in Corollary 3.1 in  [2] in 2018, as a special case when curvature information is dropped.
Yet another concern is about the experiments. Since the proposed optimal binarization has the same solution as BWN, where does the performance gain in Table 2 come from? Moreover, some of the recent quantization methods are not compared. For instance, in PACT [3], 2-bit weight&acitvation quantization already achieves 64.4% top-1 accuracy of Resnet18 on ImageNet, while the proposed method achieves the same accuracy with full-precision activation and 2-bit weight (Table 2). In addition, according to Table 2, the proposed method also can not beat LQ-Net. 
[1] Rastegari, Mohammad, et al. "Xnor-net: Imagenet classification using binary convolutional neural networks." ECCV, 2016.
[2] Hou, Lu, and James T. Kwok. "Loss-aware weight quantization of deep networks." ICLR 2018.
[3] Choi, Jungwook, et al. "Pact: Parameterized clipping activation for quantized neural networks." arXiv preprint arXiv:1805.06085 (2018).