Local Binary Pattern Networks
Memory and computation efficient deep learning architectures are crucial to the continued proliferation of machine learning capabilities to new platforms and systems, especially, mobile sensing devices with ultra-small resource footprints. In this paper, we demonstrate such an advance for the well-studied character recognition problem. We use a strategy different from the existing literature by proposing local binary pattern networks or LBPNet that can learn and perform bit-wise operations in an end-to-end fashion. Binarization of operations in convolutional neural networks has shown promising results in reducing the model size and computing efficiency. Characters consist of some particularly structured strokes that are suitable for binary operations. LBPNet uses local binary comparisons and random projection in place of conventional convolution (or approximation of convolution) operations, providing important means to improve memory and speed efficiency that is particularly suited for small footprint devices and hardware accelerators. These operations can be implemented efficiently on different platforms including direct hardware implementation. LBPNet demonstrates its particular advantage on the character classification task where the content is composed of strokes. We applied LBPNet to benchmark datasets like MNIST, SVHN, DHCD, ICDAR, and Chars74K and observed encouraging results.
In this work, a neural network that uses local binary patterns instead of kernel convolutions is introduced. Using binary patterns has two advantages: a) it reduces the network definition to a set of binary patterns (which requires much less storage than the floating point descriptions of the kernel weights used in CNNs) and b) allows for fast implementations relying only on logical operations (particularly fast on dedicated hardware).
This work is mostly descriptive of a proposed technique with no particular theoretical performance guarantees, so its value hinges mostly on its practical performance on real data. In that sense, its evaluation is relatively limited, since only figures for MNIST and SVHN are provided.
A list of additional datasets is provided in Table 5, but only the performance metric is listed, which is meaningless if it is not accompanied with figures for size, latency and speedup. The only takeway about the additional datasets is that the proposed LBPNet can match or outperform a weak CNN baseline, but we don't know if the latter achieves state-of-the-art performance (previous figures of the baseline CNN suggest it doesn't) and we don't know if there's significant gain in speed or size.
Regarding MNIST and SVHN, which are tested in some more detail, again, we are interested in the performance-speed (or size) tradeoff, and it is unclear that the current proposal is superior. The baseline CNN does not achieve state of the art performance (particularly in SVHN, for which the state-of-the-art is 1.7% and the baseline CNN achieves 6.8%). For SVHN, BCNN has a much better performance-speed tradeoff than the baseline, since it is both faster and higher performance. Then, the proposed method, LBPNet, has much higher speed, but lower performance than BCNN. It is unclear how LBPNet's and BCNN's speeds would compare if we were to match their performances. For this reason, it is unclear to me that LBPNet is superior to BCNN on SVHN.
Also the numbers in boldface are confusing, aren't they just incorrect for both the Latency and Error in MNIST? Same for the Latency in SVHN.
The description of the approach is reasonably clear and clarifying diagrams are provided. The backpropagation section seems a bit superficial and could be improved. For instance, backpropagation is computed wrt the binary sampling points, as if these were continuous, but they have been defined as discrete before. The appendix contains a bit more detail, where it seems that backpropagation is alternated with rounding. It's not justified why this is a valid gradient descent algorithm.
Also how the scaling k of the tanh is set is not explained clearly. Do you mean that with more sampling points k should be larger to keep the outputs of the approximate comparison operator close to 0 and 1?
Minor:
What exactly in this method makes it specific to character recognition? Since you are trying to capture both high-level and low-level frequencies, it seems you'd be capturing all the relevant information. SVHN data are color images with objects (digits) in it, what is the reason that makes other objects not be detectable with this approach?
English errors are pervasive throughout the paper. A non-exhaustive list:
Fig 4.b: X2 should be Y2
particuarly
"to a binary digits"
"In most case"
"0.5 possibility"
"please refer to Sec .."
"FORWARD PROPATATION"