ESPRESSO: EFFICIENT FORWARD PROPAGATION FOR BINARY DEEP NEURAL NETWORKS
There are many applications scenarios for which the computational performance and memory footprint of the prediction phase of Deep Neural Networks (DNNs) need to be optimized. Binary Deep Neural Networks (BDNNs) have been shown to be an effective way of achieving this objective. In this paper, we show how Convolutional Neural Networks (CNNs) can be implemented using binary representations. Espresso is a compact, yet powerful library written in C/CUDA that features all the functionalities required for the forward propagation of CNNs, in a binary file less than 400KB, without any external dependencies. Although it is mainly designed to take advantage of massive GPU parallelism, Espresso also provides an equivalent CPU implementation for CNNs. Espresso provides special convolutional and dense layers for BCNNs, leveraging bit-packing and bitwise computations for efficient execution. These techniques provide a speed-up of matrix-multiplication routines, and at the same time, reduce memory usage when storing parameters and activations. We experimentally show that Espresso is significantly faster than existing implementations of optimized binary neural networks (≈ 2 orders of magnitude). Espresso is released under the Apache 2.0 license and is available at http://github.com/fpeder/espresso.
The paper presents a library written in C/CUDA that features all the functionalities required for the forward propagation of BCNNs. The library is significantly faster than existing implementations of optimized binary neural networks (≈ 2 orders of magnitude), and will be released on github.
BCNNs have been able to perform well on large-scale datasets with increased speed and decreased  energy consumption, and implementing efficient kernels for them can be very useful for mobile applications. The paper describes three implementations CPU, GPU and GPU_opt, but it is not entirely clear what the differences are and why GPU_opt is faster than GPU implementation.
Are BDNN and BCNN used to mean the same concept? If yes, could you please use only one of them?
The subsection title “Training Espresso” should be changed to “Converting a network to Espresso”, or “Training a network for Espresso”.
What is the main difference between GPU and GPU_opt implementations?
The unrolling and lifting operations are shown in Figure 2. Isn’t accelerating convolution by this method a very well known one which is implemented in many deep learning frameworks for both CPU and GPU?
What is the main contribution that makes the framework here faster than the other compared work? In Figure1, Espresso implementations are compared with other implementations in (a)dense binary matrix multiplication and (b)BMLP and not (c)BCNN. Can others ( BinaryNet
(Hubara et al., 2016) or Intel Nervana/neon (NervanaSystems)) run CNNs?
6.2 MULTI-LAYER PERCEPTRON ON MNIST – FIGURE 1B AND FIGURE 1E. It should be Figure 1d instead of 1e? 
All in all, the novelty in this paper is not very clear to me. Is it bit-packing?
UPDATE: 
Thank you for the revision and clarifications. I increase my rating to 6.