Lora, Low Rank Adaptation, Generative Ai, Llm, Fine Tuning.
dimensions 100 x 5 (i.e., 500 elements); WB has dimensions 5 x 100 (i.e., 500 elements). So, we now will have to store and process 500 + 500 = 1,000 elements instead of 10,000 elements. This also means that we will not have to train the remaining 90,000 elements. That’s a huge 90% reduction in