hmmm…my understanding is that there are two of these going on.

One inside the MHA:

embed  = sf(QK/d**0.5)V

but the one here

is the following:

emebd = encoder(x) * T_x**0.5

which seems different. If you argument holds if anything I would have expected to divide by T_x**0.5 not multiply it…   though your comment gave me something to think about…