Self Attention, Deep Learning, Transformers, Bert, Ml So Good.
faster. FNet achieves 92% of the accuracy of BERT on the GLUE dataset, but it runs 7 times faster on GPU and 2 times faster on TPU. Also, there is no limitation on the input sequence length. FNet performance (Source: Paper) Final Thoughts This paper went into a new direction for optimizing the