Trasnformer, Naturallanguageprocessing, Linear Algebra.
employed in transformers, contributing to model compression and regularization. By applying SVD to weight matrices, transformers can reduce model complexity, enhance generalization, and mitigate the risk of overfitting. This paper discusses the applications of SVD in transformer training and