Avg.
65.97
73.11
60.73
34.69
26.41
35.95
39.01
27.63
19.22
33.12
31.45
33.74
35.99
36.39
36.96
39.01
33.69
35.87
42.10
43.70
23.69
39.03
55.34
58.30

Table 4. Performance of SelfExtend on Hugging Face Open LLM
benchmark suite compared with two original Llama 2 baselines.
For SelfExtend, we set the group size as 5 and the neighbor window
as 1024.
Size
7B
7B
7B
7B SelfExtend-Llama-2-chat-16k

ARC-c Hellaswag MMLU TruthfulQA GSM8k
14.33
53.24
53.32
14.10
18.95
53.07
52.56
18.42

46.30
46.32
48.32
48.34

78.51
78.54
78.41
78.43

38.96
39.00
45.24
45.33

SelfExtend-Llama 2

Name
Llama 2

Llama-2-chat-4k
Self-Extend LLM