But Low rank is for real!

They are not just about saving memory or speeding things up; they also unlock fascinating insights into the mechanistic behavior of LLMs

I can’t wait to share gradient low-rank projections (GaLore), tensor decomposition, and emerging symbolic circuits at #DeepLearn2025! 🌟