Ah I see what you are saying! M acts as a sort of permutation matrix on E. I did it on paper and see exactly what you mean now. That also brings clarity to something I’ve often read, that nn.Embedding acts as a sort of lookup table. Okay, great! This has been a big question for me for a while now. Thank you!