<p>LoRA stands for &quot;low-rank adaptation&quot;, use a <span class="math-container">$d\times d$</span> matrix, and you get rid of the &quot;LoR&quot; part in the name (aka, that you have full rank)</p>

CC BY-SA 4.0
2024-05-28T17:56:47.150Z
2024-05-28T17:56:47.150Z
Alberto
<p>I've learned that LoRA injects trainable rank decomposition 'matrices' into each layer.</p>
<p>A matrix reduces dimension from d to r, and B matrix increases dimension from r to d.</p>
<p>What I'm having trouble in understanding is that why it has this sturucture.</p>
<p>I'm curious what happens if it has simple linear strucutre from dimension d to d.</p>
<p>does the encoder - decoder structure gives some kind of feature learning??</p>

<neural-networks><fine-tuning>
Why does LoRA has encoder-decoder structure?
COTHE
# Why does LoRA has encoder-decoder structure?

(Asked by: username_0 on 2024-05-28)

I've learned that LoRA injects trainable rank decomposition 'matrices' into each layer.

A matrix reduces dimension from d to r, and B matrix increases dimension from r to d.

What I'm having trouble in understanding is that why it has this sturucture.

I'm curious what happens if it has simple linear strucutre from dimension d to d.

does the encoder - decoder structure gives some kind of feature learning??

**Tags:** neural-networks, fine-tuning


---

## 1 answers

### Answer by username_1 on 2024-05-28. Score: 0 | Accepted answer

LoRA stands for "low-rank adaptation", use a $d\times d$ matrix, and you get rid of the "LoR" part in the name (aka, that you have full rank)


---

