I’m not too sure, what exactly is T_x? If I had to guess I’d say maybe you normalize out the dimension within the self-attention mechanism but add it back in after the calculation is done to preserve the dimensionality of the input?