Deep Learning, Machine Learning, Attention Mechanism, Time Series Analysis, Coding.
queries/keys. Such locality awareness can correctly match the most relevant features based on shape matching in (left) [source] Instead of convolutional kernel size 1 with stride 1 (matrix multiplication), they employed casual convolution (to be assured that the model doesn’t have the access to the