  Graphics Processing Units (GPUs) are currently the dominating programmable
architecture for Deep Learning (DL) accelerators. The adoption of Field
Programmable Gate Arrays (FPGAs) in DL accelerators is however getting
momentum. In this paper, we demonstrate that Direct Hardware Mapping (DHM) of a
Convolutional Neural Network (CNN) on an embedded FPGA substantially
outperforms a GPU implementation in terms of energy efficiency and execution
time. However, DHM is highly resource intensive and cannot fully substitute the
GPU when implementing a state-of-the-art CNN. We thus propose a hybrid FPGA-GPU
DL acceleration method and demonstrate that heterogeneous acceleration
outperforms GPU acceleration even including communication overheads.
  Experimental results are conducted on a heterogeneous multi-platform setup
embedding an Nvidia(R) Jetson TX2 CPU-GPU board and an Intel(R) Cyclone10GX
FPGA board. The SqueezeNet, MobileNetv2, and ShuffleNetv2 mobile-oriented CNNs
are experimented. We show that heterogeneous FPG-AGPU acceleration outperforms
GPU acceleration for classification inference task over MobileNetv2 (12%-30%
energy reduction, 4% to 26% latency reduction), SqueezeNet (21%-28% energy
reduction, same latency), and ShuffleNetv2 (25% energy reduction, 21% latency
reduction).

Hardware Architecture