Figure 2. Perplexity (PPL) of Llama-2-7b-chat using grouped at-
tention on PG19 with different group size. The red dotted line
indicates the PPL of the original Llama-2-7b-chat on 4k sequence.
The purple dotted line indicates the PPL of the original Llama-2-
7b-chat on 6k sequence and it explodes.

window. Everything is the same as the original self-attention
mechanism except that before the inner product, the FLOOR
operation is applied to each token’s original position. In
Python style, this operation can be denoted as ”

Pg = P//Gs
Self-Extend LLM