medium
20.5671
8.9052
22.0958
17.6175
11.1896
7.9322
11.7518
9.3405
11.9232
15.2379
17.8975
18.5724
19.7570
8.7389
26.1203
19.9725
36.9757
29.2599
18.8173
23.0288
18.2298
23.4353
13.2253

large

18.1656
8.0454
19.3424
15.1314
10.0904
16.6742
10.8427
8.8467
10.5878
13.1190
16.4722
17.0868
17.4856
8.2928
22.6128
17.6343
28.6198
32.0796
15.9002
20.3755
15.9850
25.7138
12.9177

xl

16.9572
7.5404
17.7833
13.6267
9.3330
13.3337
10.0965
8.3238
9.8095
11.9355
16.5112
16.2709
16.7849
7.9772
20.9461
16.2905
23.4294
33.9774
14.3104
18.5649
14.6371
23.9791
11.8633

ada

16.2430
6.8800
15.4209
12.0063
7.5551
3.9614
8.7976
7.6652
9.2775
13.2112
12.5709
16.2174
12.9112
7.2458
12.1138
16.1530
6.4996
22.1295
8.4740
14.4730
16.1417
16.8190
9.7355

babbage
13.0270
5.7006
12.4220
9.5439
6.1541
3.1660
7.0366
5.9486
7.7000
10.4188
9.6349
13.8561
9.9453
6.5231
9.6995
13.1796
5.3282
17.6314
6.6394
11.6785
12.8744
13.6043
7.8456

curie
The Pile- An 800GB Dataset of Diverse Text for Language Modeling