  Systolic arrays have been widely used for accelerating HPC and deep learning
applications. There is a plethora of previous works on the performance tuning
of systolic arrays, but usually based on a number of oversimplified assumptions
(e.g., only considering divisors for loop tiling, pruning based on off-chip
data communication) to reduce the design space.
  In this paper, we present a comprehensive design space exploration tool named
Odyssey for systolic array optimization. Odyssey does not rely on artificial
assumptions to limit the design space, and yet it is highly efficient and
scalable with a hybrid optimization technique. For example, for a
1024x1024x1024 matrix multiplication, it finds designs that reach 90% of the
optimal performance in 5 seconds with a single CPU thread. Moreover, using
Odyssey, we unveil and quantify the suboptimality introduced by multiple
commonly used oversimplifications in prior studies for systolic array design
space exploration. For example, Odyssey results show that limiting to divisors
for loop tiling leads to a 39% performance loss, and pruning based on off-chip
data movement results in a 45% performance loss. We applied Odyssey to explore
the architecture trade-offs for matrix multiplication and convolutional neural
network, providing inspiration into possible optimizations for these two
applications.

Hardware Architecture