Nuclear Norm Regularization for Deep Learning
Paper
•
2405.14544
•
Published
•
1
Representation & Optimization
Understanding about representation sheds light on optimization
Note CS inequality for matrix allows penalizing element-wise Frobenius norm to encourage low-rank representations.
Token embeddings violate the manifold hypothesis
Paper • 2504.01002 • Published • 1Note Some token have more synonyms than others.
-
Approximate Nullspace Augmented Finetuning for Robust Vision Transformers
Paper • 2403.10476 • Published • 1 -
ElaLoRA: Elastic & Learnable Low-Rank Adaptation for Efficient Model Fine-Tuning
Paper • 2504.00254 • Published • 1
Flex Attention: A Programming Model for Generating Optimized Attention Kernels
Paper • 2412.05496 • Published • 1Note Customize attention mask with optimized performance comparable with Flashattention
-
Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad
Paper • 2503.21934 • Published
Value Residual Learning For Alleviating Attention Concentration In Transformers
Paper • 2410.17897 • Published • 9Note Halve KV cache via sharing value embedding across attention blocks