  State-of-the-art systolic array-based accelerators adopt the traditional
im2col algorithm to accelerate the inference of convolutional layers. However,
traditional im2col cannot efficiently support AI backpropagation.
Backpropagation in convolutional layers involves performing transposed
convolution and dilated convolution, which usually introduces plenty of
zero-spaces into the feature map or kernel. The zero-space data reorganization
interfere with the continuity of training and incur additional and
non-negligible overhead in terms of off- and on-chip storage, access and
performance. Since countermeasures for backpropagation are rarely proposed, we
propose BP-im2col, a novel im2col algorithm for AI backpropagation, and
implement it in RTL on a TPU-like accelerator. Experiments on TPU-like
accelerator indicate that BP-im2col reduces the backpropagation runtime by
34.9% on average, and reduces the bandwidth of off-chip memory and on-chip
buffers by at least 22.7% and 70.6% respectively, over a baseline accelerator
adopting the traditional im2col. It further reduces the additional storage
overhead in the backpropagation process by at least 74.78%.

Hardware Architecture