Lora, Low Rank Adaptation, Generative Ai, Llm, Fine Tuning.
compute requirements during training, without sacrificing too much of accuracy! Does it add to inference latency? You might notice in Figure 4 that at the inference time, there is a bit of inefficiency (we need to perform the addition as the last step after the data flows through the matrices)