Language Model, Transformers, Deep Learning, Artificial Intelligence, Machine Learning.
context. global Transformer that encodes the contextualized inputs. local Transformer that takes each output from the global model and predicts the output tokens in an auto-regressive manner. MEGABYTE is applied to language modeling, image modeling, and audio modeling. The cool thing is that it is