Language Model, Transformers, Deep Learning, Artificial Intelligence, Machine Learning.
MEGABYTE delivers improved performance for training and generation efficiently. MEGABYTES offers several advantages, including sub-quadratic self-attention, per-patch feedforward layers, and parallel decoding. The sub-quadratic self-attention is achieved by dividing the input into smaller