Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length
Paper
•
2404.08801
•
Published
•
68
Attention
-
-
TransformerFAM: Feedback attention is working memory
Paper • 2404.09173 • Published • 44 -
Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention
Paper • 2404.07143 • Published • 110 -
Block Transformer: Global-to-Local Language Modeling for Fast Inference
Paper • 2406.02657 • Published • 41