Self Attention, Deep Learning, Transformers, Bert, Ml So Good.
transformers. Before reading this paper, I would have never thought that Fourier transforms can replace a transformer and achieve near good performance. But the thought of viewing Fourier transforms as a way of mixing tokens is very novel. Also analyzing that weightage is not important as long as