Towards energy efficient convolutional neural network inference
Over the last few years, deep learning and particularly convolutional neural networks (CNNs) have become the method of choice for most computer vision tasks. The achieved leap in accuracy has dramatically increased the range of possibilities and meaningful application has created a demand for running these compute and memory intensive algorithms on embedded and mobile devices for reliable and costefficient deployment. In this thesis, we first evaluate the capabilities of off the shelf software programmable hardware before diving into specialized hardware accelerators and exploring the potential of extremely quantized CNNs. We give special consideration to external memory bandwidth, which dominates the overall energy cost, in the form of a conceptually different hardware architecture and with a hardware friendly de /compression scheme. We establish that including I/O softwareprogrammable platforms can achieve 10 40GOp/s/W, our specialized accelerator for fixed point CNNs achieves 630GOp/s/W, binary weight CNNs can be implemented with up to 5.9TOp/s/W and very small binarized neural networks implementable with purely combinational logic for e.g. a first classification stage could be run directly on the sensor with 670TOp/s/W. For applications in video surveillance, we further explore the effect of fusing the RGB images with multispectral data to trade off accuracy