In this paper, we introduce a method to compress intermediate feature maps of deep neural networks (DNNs) to decrease memory storage and bandwidth requirements during inference. Unlike previous works, the proposed method is based on converting fixed-point activations into vectors over the smallest GF(2) finite field followed by nonlinear dimensionality reduction (NDR) layers embedded into a DNN. Such an end-to-end learned representation finds more compact feature maps by exploiting quantization redundancies within the fixed-point activations along the channel or spatial dimensions. We apply the proposed network architecture to the tasks of ImageNet classification and PASCAL VOC object detection. Compared to prior approaches, the conducted experiments show a factor of 2 decrease in memory requirements with minor degradation in accuracy while adding only bitwise computations.
The method of this paper minimizes the memory usage of the activation maps of a CNN. It starts from a representation where activations are compressed with a uniform scalar quantizer and fused to reduce intermediate memory usage. This looses some accuracy, so the contribution of the paper is to add a pair of convolution layers in the binary domain (GF(2)) that are trained to restore the lost precision. 
Overall, this paper seems to be a nice addition to the body of works on network compression. 
+ : interesting approach and effective results. 
+ : well related to the state of the art and good comparison with other works. 
- : somewhat incremental. Most of the claimed 100x compression is due to previous work.
- : impact on runtime is not reported. Since there is a caffe implementation it would be interesting to have an additional column with the comparative execution speeds, even if only on CPU. I would expect the FP32 timings to be hard to beat, despite the claims that it uses only binary operations.
- : the paper is sometimes difficult to understand (see below)
detailed comments: 
Equations (3)-(4) are difficult to understand. If I understand correctly, b just decomposes a \hat{x} in {0..2^B-1} into its B bits \tilda{x} \in {0,1}^B, which can be then considered as an additional dimension in the activation map where \hat{x} comes from. 
It is not stated clearly whether P^l and R^l have binary weights. My understanding is that P^l has but R^l not.
4.1 --> a discussion of the large mini-batch size (1024) could be useful. My understanding is that large mini-batches are required to use averaged gradients and get smooth updates. 
end of 4.1 --> unclear what "equivalent bits" means