  It remains a challenge to run Deep Learning in devices with stringent power
budget in the Internet-of-Things. This paper presents a low-power accelerator
for processing Deep Neural Networks in the embedded devices. The power
reduction is realized by avoiding multiplications of near-zero valued data. The
near-zero approximation and a dedicated Near-Zero Approximation Unit (NZAU) are
proposed to predict and skip the near-zero multiplications under certain
thresholds. Compared with skipping zero-valued computations, our design
achieves 1.92X and 1.51X further reduction of the total multiplications in
LeNet-5 and Alexnet respectively, with negligible lose of accuracy. In the
proposed accelerator, 256 multipliers are grouped into 16 independent
Processing Lanes (PL) to support up to 16 neuron activations simultaneously.
With the help of data pre-processing and buffering in each PL, multipliers can
be clock-gated in most of the time even the data is excessively streaming in.
Designed and simulated in UMC 65 nm process, the accelerator operating at 500
MHz is $>$ 4X faster than the mobile GPU Tegra K1 in processing the
fully-connected layer FC8 of Alexnet, while consuming 717X less energy.

Hardware Architecture