NLP, Artificial Intelligence, Machine Learning, Data Science, Transformers.
reduces the computation cost. Reformer [2020]: a more efficient Transformer thanks to local-sensitive hashing attention, axial position encoding, and other optimizations. T5 [2020]: a bidirectional encoder and a left-to-right decoder pre-trained on a mix of unsupervised and supervised tasks.