Self Attention, Deep Learning, Transformers, Bert, Ml So Good.
others have cleverly approximated the self-attention matrices and sped up computations and enabled much longer sequences. But the concept of attention stayed mostly as it was. Recently many papers are coming out and rebelling against attention. No, it's not attention that we need anymore! This is