Language Model, Transformers, Deep Learning, Artificial Intelligence, Machine Learning.
Space: O(T log T + Td) 📎Paper 👩‍💻Code (unofficial) Reformer model incorporates three techniques to improve efficiency. First, it uses “reversible residuals” to reduce memory consumption by storing only one copy of the intermediate activation that can be used to reproduce the activations of the