Hi thanks for your explanation, I understand the first part now but still a bit uncertain about why this is the case.
If I understood your example correctly, if the word the’s has an embedding [0.1, -0.3, 0.2] in the embedding matrix, and if the output of the decoder (before feeding it into the lm head) have a vector [0.099, =0.302, 0.18], we would want to predict something close to the. What I don’t understand however, is why using the transposed embedding matrix would be better than initialising another matrix to achieve this. What do you mean by only having to learn one representation of embedding vectors?
Thanks in advance!