Lora, Low Rank Adaptation, Generative Ai, Llm, Fine Tuning.
of a matrix isn’t necessarily same as its dimensions. Rather, it is equal to the number of linearly independent rows or columns. That’s why we can represent the large matrix of delta weights as a combination of potentially smaller matrices through the process called matrix decomposition, where ΔW =