Low Rank Sparse Attention
Collection
Open source weights of Lorsa modules introduced in "Towards Understanding the Nature of Attention with Low-Rank Sparse Decomposition".
•
3 items
•
Updated
•
2