Lora, Low Rank Adaptation, Generative Ai, Llm, Fine Tuning.
delta weights separately instead of combining them with the weights, because LoRA will take advantage of this representation. In mathematical terms, h = W’x = (W+ΔW)x = Wx + ΔWx. Figure 2. Alternative representation of fine-tuning for easier explanation of LoRA Let’s take a pause here and