Massive Activations in Hybrid Linear Attention Large Language Models: Pre-Attention Spikes and Inter-Spike Plateaus
Paper • 2608.12149 • Published • 30
AI & ML interests
None defined yet.
Recent Activity
View all activity
Lie24
authored a
paper 7 days ago
zunhai
authored 3
papers 8 days ago
Massive Activations in Hybrid Linear Attention Large Language Models: Pre-Attention Spikes and Inter-Spike Plateaus
Paper • 2608.12149 • Published • 30
SnapMLA: Efficient Long-Context MLA Decoding via Hardware-Aware FP8 Quantized Pipelining
Paper • 2602.10718 • Published
XStreamVGGT: Extremely Memory-Efficient Streaming Vision Geometry Grounded Transformer with KV Cache Compression
Paper • 2601.01204 • Published
zunhai
submitted a
paper to Daily Papers 8 days ago
zunhai
authored a
paper 9 days ago
Lie24
authored a
paper 3 months ago
zunhai
authored a
paper 3 months ago
zunhai
submitted a
paper to Daily Papers 3 months ago
zunhai
authored a
paper 4 months ago
zunhai
submitted a
paper to Daily Papers 4 months ago
Lie24
authored a
paper 4 months ago
Shuibai12138
authored a
paper 5 months ago
Shuibai12138
submitted a
paper to Daily Papers 5 months ago
zunhai
authored a
paper 7 months ago
zunhai
authored 4
papers 9 months ago
RotateKV: Accurate and Robust 2-Bit KV Cache Quantization for LLMs via Outlier-Aware Adaptive Rotations
Paper • 2501.16383 • Published
AKVQ-VL: Attention-Aware KV Cache Adaptive 2-Bit Quantization for Vision-Language Models
Paper • 2501.15021 • Published
LongCat-Flash Technical Report
Paper • 2509.01322 • Published • 10
DoPE: Denoising Rotary Position Embedding
Paper • 2511.09146 • Published • 98
zunhai
authored a
paper about 1 year ago