Deep neural network (DNN) has rapidly found many applications in different scenarios. However, its large computational cost and memory consumption are barriers to computing restrained applications. DNN model quantization is a widely used method to reduce the DNN storage and computation burden by decreasing the bit width. In this paper, we propose a novel cursor based adaptive quantization method using differentiable architecture search (DAS). The multiple bits' quantization mechanism is formulated as a DAS process with a continuous cursor that represents the possible quantization bit. The cursor-based DAS adaptively searches for the desired quantization bit for each layer. The DAS process can be solved via an alternative approximate optimization process, which is designed for mixed quantization scheme of a DNN model. We further devise a new loss function in the search process to simultaneously optimize accuracy and parameter size of the model. In the quantization step, based on a new strategy, the closest two integers to the cursor are adopted as the bits to quantize the DNN together to reduce the quantization noise and avoid the local convergence problem. Comprehensive experiments on benchmark datasets show that our cursor based adaptive quantization approach can efficiently obtain lower size model with comparable or even better classification accuracy.
The paper proposes a novel way to determine automatically the quantization level at each layer of 
deep neural networks while training. The quantization can result in a more compact network requiring
less memory for storing the parameters (weights), with minimal drop in accuracy. 
If I understand correctly, the main idea in the paper is utilizing a continuous representation of the quantization using a continuous cursor, and a new differential approach from Neural Architecture Search (NAS), to create a differential learning algorithm for the problem of finding the optimal quantization. 
On the positive side, the authors present very good experimental results on popular networks like ResNet, 
and improve on other method's compression accuracy, with little to no reduction in accuracy. 
Howerver, the paper is not clearly and carefully enough written to convey the author's results which makes it a borderline paper in my opinion. 
In my understanding, compressed (quantized) networks can have different advantages besides a more compact representation of the network: 
For example, this can also reduce training time because all operations are performed with words with fewer bits - but the authors focus only on the compression ratio and do not discuss this issue. In fact, it seems that their algorithm actually requires more training time than standard training of neural networks, because you start with training w with full 32-bit representation, and then train with different quantizations simultaneously. 
It would be good if the authors describe clearly how do different methods (theirs and the others mentioned) compare to each other in terms of savings of different resources (training time, compression, inference/prediction time for the trained networks etc.)  and which of these is more important (e.g. the latter in mobile devices).
Another practical issue is that using a different number of bits for each layer may complicate the design and software/hardware implementation of a network, compared to say allocating 4-bits or 8-bits to each weight. It is correct that you can reduce the overall number of bits, but you may reduce it even further if you allow a different number of bits for each individual weight. I assume that there is a price in power/memory/speed etc. to this non-homogenous property, and it may be better to get a slightly lower compression ratio provided that there is more uniformity in the network weights.  This is not a critique of this particular paper, but of the entire framework of flexible quantization. 
The English of the manuscript could be greatly improved. For example: 
Page 2: "..multiple bits for different layers.." - I assume the authors mean "..a different number of bits used for different layers .."
Page 2, bottom: "The authors (Wang et al. 2018) presented ..." - the sentence is long and unclear. 
Many sentences starting with 'And' - for example: "And a Gumbel softmax function ..." (page 3) 
There are a few vague and non-informative statements which do not contribute to the understanding of the field and the authors' contribution. For example: 
- Page 3: "A typical search method is random search, however, its efficiency is not ideal." (does efficiency refer to computational efficiency here?)
- Page 4: "The reason why we add regularization item to the loss function is because the regularization can prevent overfitting to some extent" (I thought that here the main purpose of regularization is to get a trade-off between accuracy and compression). 
- Page 6: Regarding parameters choice: 'a rather optimal set of them is chosen ..' - this seems quite arbitrary and it is not clear how to choose parameters for the authors' method for future architectures and datasets. 
There are also terms that are known to expert but could use a short explanation/reference - for example: 
- Adam optimizer 
- Cosine annealing
- Gumble Softmax
The mathematical equations are definitions are not clear enough and contain errors: 
- Eq. (1): there is (x',y') twice in the equation, but in the sentence afterward there is also (x,y). 
There are D_T, D_V with and without tilde. 
The space of maximization of w for a given quantization is not defined (I'm assuming all weights vectors w
which can be represented using the number of bits in the quantization). 
- Eq. (3) - the loss is loosely defined - what is 'parameter size'? the number of bits for all the weights in each layer? 
- Eq. (4) - is w or w_k the full precision weights? why are x and y in the interval [0,1]?
- Eq. (6) - c_i represnts the i-th layer, but d_1,d_2, a_1,a_2 are fixed. Are they also different for different layers?   
Also, a_1 and a_2 are the boundaries of the cursor - are they set to [0,1]? or [0,32]? 
- The authors defined Conv - the convolution operation - but what are W_1 and X? vectors? how is the convolution
defined precisely? it is also not common in math papers to use '*' for  product
It is not entirely clear to me how exactly the authors define a differentiable loss and a gradient for the quantization part Loss_q.  When a cursor parameter changes (e.g. c_i from 2.5 to 2.6) then the loss is defined by quantization to the two consecutive integers (in that case 2,3 ) and there is a continuous mixture parameter between them which can be changed smoothly, which is nice. But at some point, the parameter will reach 3, and then the quantization will use the two integers 3 and 4 - the fact that 3 has weight 1 in the loss (and 2,4 have weight zero) makes the loss continuous, but is the loss differentiable at this point?  some explanation is needed here on if this is a problem and how is it avoided/solved. 
Algorithm 1: The description in Figure 1 can be made more precise. The algorithm seems to alternate between 
optimizing the cursor c, and optimizing the weights w for a given value of c. 
There are also missing details and parameters like gradient step size.  What does Grad-C * L_C mean? is the gradient multiplied by the loss?? or only applied to it? 
In the sentence "Quantize the network ... to update the loss" - which loss is updated and how? L_V, L_T? something else? 
A detailed description of the algorithm and parameters or the actual code used by the authors would improve the understanding of their method. 
Section 4.1: Figure 2 compares the loss of one-integer vs. two-integers quantization scheme. The authors argue that their two integers scheme is better because it is smoother. But the loss for one integer is actually lower - so why wouldn't it be better to use this one?  
will the two-integer method eventually reach a lower loss?