Artificial Intelligence, Deep Learning, Machine Learning, Neural Networks, Math.
single k_j on a given q_i​ is diminished because it’s averaged over a greater number of terms: With larger n, the denominator grows larger, spreading the attention more thinly across more tokens. This “dilution” reduces the ability of the model to focus sharply on the most relevant tokens. Increase