I am wondering how to set the values in the module, i.e. I tried it but got errors (cryptic) that the dimensions do not match.
I looked at the source code, now I am wondering if I misunderstand the implementation or if there is an error in the implementation? The dimension for the q embedding should be the same as kdim in my opinion (in red), as q and v are used for calculating the correlations. Why is it set to embed_dim?

image1360×1122 164 KB