Ultra-Sparse Memory Network
Paper
•
2411.12364
•
Published
•
24
Full Paper List
-
-
Hyper-Connections
Paper • 2409.19606 • Published • 23 -
Polynomial Composition Activations: Unleashing the Dynamics of Large Language Models
Paper • 2411.03884 • Published • 29 -
Over-Tokenized Transformer: Vocabulary is Generally Worth Scaling
Paper • 2501.16975 • Published • 31 -
Scale-Distribution Decoupling: Enabling Stable and Effective Training of Large Language Models
Paper • 2502.15499 • Published • 13 -
HybridNorm: Towards Stable and Efficient Transformer Training via Hybrid Normalization
Paper • 2503.04598 • Published • 20 -
Frac-Connections: Fractional Extension of Hyper-Connections
Paper • 2503.14125 • Published • 21 -
Efficient Pretraining Length Scaling
Paper • 2504.14992 • Published • 19 -
Scaling Law for Quantization-Aware Training
Paper • 2505.14302 • Published • 62