Previous ternarizations such as the trained ternary quantization (TTQ), which quantized weights to three values (e.g., {−W n , 0, +W p }), achieved the small model size and efficient inference process. However, the extreme limit on the number of quantization steps causes some degradation in accuracy. To solve this problem, we propose a hybrid weight representation (HWR) method which produces a network consisting of two types of weights, i.e., ternary weights (TW) and sparse-large weights (SLW). The TW is similar to the TTQ's and requires three states to be stored in memory with 2 bits. We utilize the one remaining state to indicate the SLW which is referred to as very rare and greater than TW. In HWR, we represent TW with values while SLW with indices of values by encoding SLW. As a result, the networks can preserve their model size with improving their accuracy compared to ternary weights. To sparsify non-ternary weights in quantization, we also introduce a centralized quantization (CQ) process with a weighted ridge (WR) regularizer. They aim to reduce the entropy of weight distributions by centralizing weights toward ternary values. Our comprehensive experiments show the efficiency of HWR in terms of the trade-off between model size and accuracy.
This paper is about quantization, and how to represent values as the finite number of states in a low bit width, using discretization. Particularly, they propose an approach to tackle the problems associated with previous ternarization which quantize weights to three values. Their approach is a hybrid weight representation method, which uses a network to output two weight types: ternary weight and sparse-large weights. For the ternary weight, they need 3 states to be stored with 2 bits. The one remaining state is used to indicate the sparse-large weight. They also propose an approach to centralize the weights towards ternary values. Their experiments show that their approach outperforms other compressed modeling approaches, and show an increase in AlexNet performance on the CIFAR-100 while increasing model size by 1.13%. 
- Overall, this is an interesting paper, offering a novel solution to tackle the degradation in accuracy occuring in ternary quantization techniques because of the number of quantization steps.  Their method seems technically sound, however I am not familiar with this area, so I would trust more the opinion of other reviewers - subject experts in the matter.
- Their results on AlexNet and ResNet do show an improvement in terms of model accuracy, with only a slight increase in model size. They have also provided extensive experiments studying the benefits of their quantization method, the tradeoff among accuracy and model size. 
- I find the abbreviations, being used too often, to be confusing
- The paper is targeted for a very focused audience, and does not give enough background for readers not familiar with "ternarizations".  Even the abstract could benefit from a motivation/ problem statement sentence, as well as less abbreviations being used. 
I would vote for acceptance of this paper, although it does seem a too-targeted paper on a specific audience. I would urge the authors to revise writing to make it broader accessible.