Language Model, Transformers, Deep Learning, Artificial Intelligence, Machine Learning.
sub-quadratic, hardware-efficient, and expressive.” In Monarch Mixer, they use layers built up from Monarch matrices to mix across the sequence (replacing the Attention operation) and across the model dimension (replacing the dense MLP). Conformers 📎 Paper 👩‍💻 Code (unofficial) The Conformer is a