McDRAM: Low Latency and Energy Efficient Matrix Computations in DRAM
We propose a novel memory architecture for in memory computation called McDRAM, where DRAM dies are equipped with a large number of multiply accumulate (MAC) units to perform matrix computation for neural networks. By exploiting high internal memory bandwidth and reducing off chip memory accesses, McDRAM realizes both low latency and energy efficient computation. In our experiments, we obtained the chip layout based on the state of the art memory, LPDDR4 where McDRAM is equipped with 2048 MACs in a single chip package with a small area overhead (4.7% Compared with the state of the art accelerator, TPU and the power efficient GPU, Nvidia P4, McDRAM offers <inline formula> <tex math notation="LaTeX"$9.5{\times /tex math>/inline formula> and <inline formula> <tex math notation="LaTeX"$14.4{\times /tex math>/inline formula> speedup, respectively, in the case that the large scale MLPs and RNNs adopt the batch size of 1. McDRAM also gives <inline formula> <tex math notation="LaTeX"$2.1{\times /tex math>/inline formula> and <inline formula> <tex math notation="LaTeX"$3.7{\times /tex math>/inline formula> better computational efficiency in TOPS/W than TPU and P4, respectively, for the large batches.
IEEE Transactions on Computer Aided Design of Integrated Circuits and Systems