Jamba: A Hybrid Transformer-Mamba Language Model
Paper
•
2403.19887
•
Published
•
111
LLM
-
-
Aurora-M: The First Open Source Multilingual Language Model Red-teamed according to the U.S. Executive Order
Paper • 2404.00399 • Published • 43 -
Mixture-of-Depths: Dynamically allocating compute in transformer-based language models
Paper • 2404.02258 • Published • 106 -
Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length
Paper • 2404.08801 • Published • 68 -
Multi-Head Mixture-of-Experts
Paper • 2404.15045 • Published • 61 -
Octopus v4: Graph of language models
Paper • 2404.19296 • Published • 119 -
Layerwise Recurrent Router for Mixture-of-Experts
Paper • 2408.06793 • Published • 33 -
Amuro & Char: Analyzing the Relationship between Pre-Training and Fine-Tuning of Large Language Models
Paper • 2408.06663 • Published • 16